Skip to main content
  • Log in
  • Manage Cookies
Eclipse Foundation
Download
  • Projects
  • Working Groups
  • Members
  • Community
    • Marketplace
    • Events
    • Planet Eclipse
    • Newsletter
    • Videos
    • Blogs
  • Participate
    • Report a Bug
    • Forums
    • Mailing Lists
    • Wiki
    • IRC
    • Research
  • Eclipse IDE
    • Download
    • Learn More
    • Documentation
    • Getting Started / Support
    • How to Contribute
    • IDE and Tools
    • Newcomer Forum
  • More
      • Community

      • Marketplace
      • Events
      • Planet Eclipse
      • Newsletter
      • Videos
      • Blogs
      • Participate

      • Report a Bug
      • Forums
      • Mailing Lists
      • Wiki
      • IRC
      • Research
      • Eclipse IDE

      • Download
      • Learn More
      • Documentation
      • Getting Started / Support
      • How to Contribute
      • IDE and Tools
      • Newcomer Forum
    • Search

  1. Home
  2. Blogs
  3. Mikaël Barbero's blog
  4. Scaling up the Continuous Integration infrastructure for Eclipse Foundation’s projects — Act 2

Scaling up the Continuous Integration infrastructure for Eclipse Foundation’s projects — Act 2

Tuesday, May 29, 2018 - 04:00 by Mikaël Barbero

TL;DR

Infrastructure improvements and migration described in last year post is eventually happening, with some tweaks.

As of today, more than 250 Eclipse projects use the build infrastructure at the Eclipse Foundation. For a year now, we’re planning how the infrastructure can be scaled and expanded to keep up with the high demand. Improving the utilization and efficiency of our current hardware and integrating new hardware and cloud resources are the main goals of this effort.

About a year ago, we announced that we were planning to migrate the entire build infrastructure at the Eclipse Foundation to CloudBees Core (formerly known as CloudBees Jenkins Enterprise - CJE) on top of RedHat OpenShift Container Platform, a Kubernetes distribution. What happened since then? CloudBees Core has been setup last June and we eventually created all JakartaEE Jenkins instances (and a couple more) on it until last October. While hitting a couple of issues in the beginning, the whole stack allowed us to provide just enough computing resources to the ~40 JakartaEE projects and allowed Glassfish 5.1 to be build on the Eclipse infrastructure and released a couple of weeks ago. Still, we are not entirely satisfied with the current state, and a couple of showstoppers prevent us from migrating all existing Jenkins instances (the ones on ci.eclipse.org) to CloudBees Core. Here are two examples of such showstoppers:

  • First, we cannot create CloudBees Core Jenkins instances in a way that would allow us to setup resource quota management per project. It means, that the whole cluster would be available to all projects and the issue we have on our current infrastructure (one project can starve others) would be even worse: it would happen at the cluster level, impacting all projects and not only the ones sharing the same machine of the greedy project. 
  • The second main issue is about secret management, more specifically, secrets that cannot be stored in Jenkins credentials. On the current infrastructure, secrets are kept on the file system and we use good old POSIX permissions to prevent projects to read other projects’ secrets. With CloudBees Core, all Jenkins instances run as the same cluster user and thus can read secrets from others (I’m oversimplifying here, if you want details, feel free to reach out to us on cbi-dev@eclipse.org mailing list). It is highly undesirable.

We’ve worked with CloudBees to find solutions and/or workarounds, but of course we’re not their only customer and our requirements are not necessarily top priorities for their products. It was a bit frustrating because we knew that most of the issues could be solved by interacting with Kubernetes directly (OpenShift more specifically in our case). Instead we had to go through CloudBees Core UI/CLI which was abstracting the cluster management and did not provide all the options we needed. Eventually, we had to move on. We learned a lot about Kubernetes/OpenShift in the meantime, and we tested what would be the result if we were deploying the Open Source version of Jenkins in the cluster. Surprisingly it worked very well and the freedom we get by having a hand on every deployment aspects really felt liberating. 

So here we are. We now have a tailored, organic, home-grown solution to manage Jenkins instances in the new clustered infrastructure and we’re about to start the migration. The first instance to be migrated will be the CBI one and it will happen in the next couple of days. We will also start reaching out to projects to announce when the migration will happen. Don’t call us, we will call you, very soon :). We don’t expect much disruption, and most projects will only need to change minor things in their build settings. You can track the global effort with the top level ticket #544221 on Bugzilla.

We have setup a Migration FAQ on the Eclipse wiki. Feel free to let us know if you have any concerns or questions in the meantime.

As part of this effort, we would like to thank both CloudBees and RedHat for their generous donations in the form of software and support. We especially thank CloudBees for their help and support. Unfortunately, a custom tailored solution to our very specific requirements (feature-wise and time-wise) will be a better fit than CloudBees Core.

Source: 
https://mikael.barbero.tech/blog/post/scaling-up-ci-infrastructure-act2/
  • Mikaël Barbero's blog

Eclipse Foundation Blogs

  • Wayne Beaton (820 posts)
  • Mike Milinkovich (319 posts)
  • Ivar Grimstad (244 posts)
  • Benjamin Cabé (131 posts)
  • Tanja Obradovic (60 posts)
  • Thabang Mashologu (37 posts)
  • John Kellerman (27 posts)
  • Paul Buck (22 posts)
  • Brian King (19 posts)
  • Frédéric Desbiens (19 posts)
  • Christopher Guindon (15 posts)
  • Gael Blondelle (14 posts)
  • Mikaël Barbero (14 posts)
  • Hailley Seed (10 posts)
  • Denis Roy (9 posts)
  • Hudson Kelly (8 posts)
  • Michael Plagge (4 posts)
  • Serina El Salibi (3 posts)
  • Shabnam Mayel (3 posts)
  • Shanda Giacomoni (3 posts)
  • Jacob Harris (2 posts)
  • Clark Roundy (2 posts)
  • Paul White (1 posts)
  • Stephanie Swart (1 posts)
  • Karla Ferrer (1 posts)
  • Sharon Corbett (1 posts)

Recent blog posts

  • Hashtag Jakarta EE #162
  • DEVIES Award to Jakarta EE 10
  • Jakarta EE track at Devnexus 2023!!!!
  • Hashtag Jakarta EE #161
  • Jakarta EE Community Update - 2022 in Review
  • jChampionsConf 2023
  • Eclipse Cloud DevTools Contributor Award: Theia Developers for Detachable Views
  • Hashtag Jakarta EE #160
  • THAT Conference 2023
  • European Cyber Resiliency Act: Potential Impact on the Eclipse Foundation
More

Eclipse Foundation

  • About Us
  • Contact Us
  • Sponsor
  • Members
  • Governance
  • Code of Conduct
  • Logo and Artwork
  • Board of Directors
  • Careers

Legal

  • Privacy Policy
  • Terms of Use
  • Copyright Agent
  • Eclipse Public License
  • Legal Resources

Useful Links

  • Report a Bug
  • Documentation
  • How to Contribute
  • Mailing Lists
  • Forums
  • Marketplace

Other

  • IDE and Tools
  • Projects
  • Working Groups
  • Research@Eclipse
  • Report a Vulnerability
  • Service Status

Copyright © Eclipse Foundation. All Rights Reserved.

Back to the top