Automatic text processing is something that is widely used across the web, whether it’s used to understand what you type in a search bar or detect spam in email. Developing fast and efficient text classification, however, is something that has challenged AI researchers throughout the years, and that’s why the Facebook AI Research team (FAIR) set out the develop fastText. FAIR has taken things one step further with the fastText, today announcing that it will become open-source.
FastText is a library that aims to assist in the creation of “scalable solutions for text representation and classification.” One of the major strengths of fastText is that it was designed to help with text processing as it applies to very large databases, something that deep neural networks – which are another popular solution when it comes to text classification – can have trouble with. As FAIR points out, deep-learning methods are usually slow to train and test.
FastText is capable of training on large databases in a matter of seconds or minutes, as opposed to the hours or days it can take deep-learning based methods. Something like fastText can already serve as a crucial component to the spam filters employed across the web, but it could also be a huge help as we move into the future, as the speed fastText provides can help AI bots like Siri and Google Now better parse natural language more efficiently.
Open-sourcing fastText, FAIR says, is part of the team’s “ongoing commitment to collaboration and sharing with the community.” As part of that commitment, FAIR has also shared its own research on fastText, giving those who want to help improve it or use it for their own applications a springboard to getting started. Those interested in learning more about fastText can click the source link below, while FAIR’s upload of fastText’s code can be found on GitHub.
SOURCE: Facebook AI Research