Machine Learning with Spark - Second Edition by Nick Pentreath

Key FeaturesGet to the grips with the latest version of Apache SparkUtilize Sparks machine learning library to implement predictive analyticsLeverage Sparks powerful tools to load, analyze, clean, and transform your dataBook DescriptionThis book will teach you about popular machine learning algorithms and their implementation. You will learn how various machine learning concepts are implemented in the context of Spark ML. You will start by installing Spark in a single and multinode cluster. Next youll see how to execute Scala and Python based programs for Spark ML. Then we will take a few datasets and go deeper into clustering, classification, and regression. Toward the end, we will also cover text processing using Spark ML.Once you have learned the concepts, they can be applied to implement algorithms in either green-field implementations or to migrate existing systems to this new platform. You can migrate from Mahout or Scikit to use Spark ML.By the end of this book, you will acquire the skills to leverage Sparks features to create your own scalable machine learning applications and power a modern data-driven business.What You Will LearnGet hands-on with the latest version of Spark MLCreate your first Spark program with Scala and PythonSet up and configure a development environment for Spark on your own computer, as well as on Amazon EC2Access public machine learning datasets and use Spark to load, process, clean, and transform dataUse Sparks machine learning library to implement programs by utilizing well-known machine learning modelsDeal with large-scale text data, including feature extraction and using text data as input to your machine learning modelsWrite Spark functions to evaluate the performance of your machine learning modelsAbout the AuthorRajdeep Dua has over 16 years of experience in the Cloud and Big Data space. He worked in the advocacy team for Googles big data tools, BigQuery. He worked on the Greenplum big data platform at VMware in the developer evangelist team. He also worked closely with a team on porting Spark to run on VMwares public and private cloud as a feature set. He has taught Spark and Big Data at some of the most prestigious tech schools in India: IIIT Hyderabad, ISB, IIIT Delhi, and College of Engineering Pune.Currently, he leads the developer relations team at Salesforce India. He also works with the data pipeline team at Salesforce, which uses Hadoop and Spark to expose big data processing tools for developers. He has published Big Data and Spark tutorials at http://www.clouddatalab.com. He has also presented BigQuery and Google App Engine at the W3C conference in Hyderabad ( http://wwwconference.org/proceedings/www2011/schedule/www2011_Program.pdf). He led the developer relations teams at Google, VMware, and Microsoft, and he has spoken at hundreds of other conferences on the cloud. Some of the other references to his work can be seen at http://yourstory.com/2012/06/vmware-hires-rajdeep-dua-to-lead-the-developer-relations-in-india/ and http://dl.acm.org/citation.cfm?id=2624641.His contributions to the open source community are related to Docker, Kubernetes, Android, OpenStack, and cloudfoundry.You can connect with him on LinkedIn at https://www.linkedin.com/in/rajdeepd.Manpreet Singh Ghotra has more than 12 years of experience in software development for both enterprise and big data software. He is currently working on developing a machine learning platform using Apache Spark at Salesforce. He has worked on a sentiment analyzer using the Apache stack and machine learning.He was part of the machine learning group at one of the largest online retailers in the world, working on transit time calculations using Apache Mahout and the R Recommendation system using Apache Mahout.With a masters and postgraduate degree in machine learning, he has contributed to and worked for the machine learning community.His GitHub profile is https://github.com/badlogicmanpreet and you can find him on LinkedIn at https://in.linkedin.com/in/msghotra.Nick Pentreath has a background in financial markets, machine learning, and software development. He has worked at Goldman Sachs Group, Inc., as a research scientist at the online ad targeting start-up, Cognitive Match Limited, London, and led the data science and analytics team at Mxit, Africas largest social network.He is a cofounder of Graphflow, a big data and machine learning company focused on user-centric recommendations and customer intelligence. He is passionate about combining commercial focus with machine learning and cutting-edge technology to build intelligent systems that learn from data to add value to the bottom line.Nick is a member of the Apache Spark Project Management Committee.Table of ContentsGetting Up and Running with SparkMath for Machine LearningDesigning a Machine Learning SystemObtaining, Processing, and Preparing Data with SparkBuilding a Recommendation Engine with SparkBuilding a Classification Model with SparkBuilding a Regression Model with SparkBuilding a Clustering Model with SparkDimensionality Reduction with SparkAdvanced Text Processing with SparkReal-Time Machine Learning with Spark StreamingPipeline APIs for Spark ML