Serverless services are those that fit into the classes of backends as a service (BaaS) (vendor-hosted, highly scalable data and logic components) and functions as a service (FaaS) (the ability to write custom server-side software in small, event-triggered functions, deployed to a fully managed platform). Serverless architectures are those systems that make significant use of either or both of these types of service. Data pipelines are an excellent use case for serverless architecture because of their inherently event-driven behavior and often volatile nature of scaling.
Mike Roberts describes a real-life example where an existing data platform was rearchitected and reengineered to provide several improvements—significantly increased data capacity, reduced cost, and vastly improved development cycle time—all while reducing operational effort. Such gains were possible by using the vast array of serverless services provided by AWS, such as Kinesis, Lambda, and S3. At the same time, “always on” server components were included within the pipeline where appropriate to create a hybrid architecture of serverless and traditional architecture.
Building such an architecture is just part of the story, of course; running it without headaches is just as crucial. Therefore activities like continuous delivery and monitoring also need to be part of the picture. Mike explains how to implement these using further AWS services, including CodePipeline, Serverless Application Model, and CloudFormation. You’ll learn what is possible when building and running scalable serverless data pipelines on AWS and leave ready to start your own engineering of such systems.
Mike Roberts is a partner at Symphonia, a cloud technology consultancy based in New York City. Mike is a longtime proponent of Agile and DevOps values and is excited by the role that cloud technologies have played in enabling such values for many high-functioning software teams. Mike can be reached at firstname.lastname@example.org.
©2017, O’Reilly UK Ltd • (800) 889-8969 or (707) 827-7019 • Monday-Friday 7:30am-5pm PT • All trademarks and registered trademarks appearing on oreilly.com are the property of their respective owners. • email@example.com