How do you connect a Hadoop cluster to an enterprise directory with 100,000+ users and centralized role and access management?
ING has separate security architectures for Unix-based and Windows-based systems. User keys and group memberships can differ in both worlds and are technologically managed by different architecture and support groups of the organization. ING’s security architecture for Hadoop relies on Kerberos for authentication and Apache Ranger for authorization.
When the security architecture was created, the (Windows) Active Directory was used as the authentication point, creating a number of challenges. It required cooperation of the Windows and Linux teams to set up connections and trust relationships. Solutions had to be found to map Windows identities to Linux IDs, which are required for specific Hadoop components to work. Apache Ranger uses LDAP queries for synchronization but these queries scale poorly when the user base is big (100,000+ users). The team also had to figure out how to manage and control keytab files on a Kerberized Hadoop cluster in a safe way and how to manage keytab files on systems that are not managed by Ambari.
Hellmar Becker and Frank Albers present ING’s approach to aligning Hadoop authentication and role management with ING’s policies and architecture, discuss challenges they met on the way, and outline the solutions they found.
Hellmar Becker is a solutions engineer at Hortonworks, where he is helping spread the word about what you can do with data in the modern world. Hellmar has worked in a number of positions in big data analytics and digital analytics. Previously, he worked at ING Bank implementing the Datalake Foundation project (based on Hadoop) within client information management.
Frank Albers is a software engineer on the big data DevOps team at ING. He specializes in HortonWorks/Hadoop, infrastructure solution patterns for cloud services, architecture, migration to the (private) cloud, security, and networking.
Comments on this page are now closed.
©2016, O’Reilly UK Ltd • (800) 889-8969 or (707) 827-7019 • Monday-Friday 7:30am-5pm PT • All trademarks and registered trademarks appearing on oreilly.com are the property of their respective owners. • email@example.com
Apache Hadoop, Hadoop, Apache Spark, Spark, and Apache are either registered trademarks or trademarks of the Apache Software Foundation in the United States and/or other countries, and are used with permission. The Apache Software Foundation has no affiliation with and does not endorse, or review the materials provided at this event, which is managed by O'Reilly Media and/or Cloudera.