At our recent FCIA webcast “Benefits of FC-NVMe for Containerized ML Models,” it was quite clear that NVMe over Fibre Channel reduces the training time for container Machine Learning/Deep Learning workloads, and the robust reliability, performance and full offload of Fibre Channel (FC) make it an ideal choice for AI/ML. The live session generated several interesting questions from the audience and our FCIA experts have answered them here:
Q. What was the type of data and algorithm used in in the suggested solution?
A. We used up to 35GB of textual unstructured data as our training data which was collected from our internal database. We removed symbols, spaces, punctuation etc for cleaning and ran the modeling over the cleaned training data. After a couple of rounds of data cleaning and pre-processing, we used two main modeling algorithms: The first being the word2vec model followed by Tf-idf model. These models gave us the best accuracy in several natural language processing (NLP) models we tried for our data.
Q. Does the ML algorithm depend on the type of data?
A. Yes, the ML algorithm depends on the type of training data. For example, we have textual data hence we are using NLP algorithms, if we had image or streaming data, we could use fuzzy logic etc.
Q. Will the performance improvement be the same regardless of the training data type?
A. Yes, as expected the performance improvement is seen regardless of the training data type.
Q. Are there any limitations of using Fibre Channel with containers when using ML Workloads?
A. No, Fibre Channel is included in the Linux distributions. We suggest you follow the documentation from the OS vendor on Fibre Channel usage.
Q. Did you use point to point connections for FC or did you have switches in between?
A. We used switches in between, we did not connect directly to the storage array interfaces. Typically, FC configurations include switches, so point to point connections were not used.
Q. What types of FC switches were used and did you try different types of switches?
A. We used Brocade switches for this proposed solution; however, we have used Cisco switches as well in the past for this solution.
If you missed this presentation, it is available on-demand on BrightTALK and on the FCIA YouTube Channel. We hope you will take the time to view it at your convenience and share it with your colleagues. The presentation slides are also available and can be downloaded here.
