With the rapid development and evolution of applications, high data volumes, and users influx, there’s a need to develop intelligent systems that can assist in data analysis and decision making. Flavio Clesio and Eiti Kimura offer a practical demonstration of using machine learning to create an intelligent monitoring application based on a distributed system data analysis using Apache Spark MLlib.
Monitoring distributed systems usually is a tricky task. Flavio and Eiti share their experience implementing machine-learning techniques in the development of a data analysis application to monitor a distributed platform responsible for charging user subscriptions at mobile carriers in Brazil. The application, Watcher-AI, uses linear regression algorithms with Apache Spark (MLlib) to make a forecast and check if the platform is experiencing any operational problems. Flavio and Eiti used Scala for processing and training machine-learning models and then developed a Java application that uses these models to predict the expected outputs. Watcher-AI is able to detect deviations in charging numbers and provides notifications stating the problem based on the numbers or on the platform so that users can work quickly to avoid serious problems that directly impact the company’s revenues and reduce the time for action.
Flavio Clesio is specialist in machine learning and revenue assurance at Movile, where he helps build core intelligent applications to exploit revenue opportunities and automation in decision making. Prior to Movile, Flavio was a business intelligence consultant in financial markets, specifically in nonperforming loans. He holds a master’s degree in computational intelligence applied in financial markets.
Eiti Kimura is an IT coordinator and architect of distributed and high-performance platforms at Movile Brazil. Eiti has over 15 years of experience working with software development. He is an enthusiast of open technologies—he was an Apache Cassandra MVP from 2014 to 2015—and has vast experience with backend systems for carrier billing services, sending bulk text messages (SMS), and user action tracking. Eiti holds a master’s degree in electrical engineering with a specialization in software engineering.
Comments on this page are now closed.
©2016, O'Reilly Media, Inc. • (800) 889-8969 or (707) 827-7019 • Monday-Friday 7:30am-5pm PT • All trademarks and registered trademarks appearing on oreilly.com are the property of their respective owners. • firstname.lastname@example.org
Apache Hadoop, Hadoop, Apache Spark, Spark, and Apache are either registered trademarks or trademarks of the Apache Software Foundation in the United States and/or other countries, and are used with permission. The Apache Software Foundation has no affiliation with and does not endorse, or review the materials provided at this event, which is managed by O'Reilly Media and/or Cloudera.