Installing a Package in an Openshift Container (Kind Of)
07.25.2017
TODO Better title. Ping me with ideas!
Fair warning: This article assumes that you have a decent understanding of Openshift from the outset, because explaining all the concepts used here would take too much space. I'll try to link to resources where applicable though, so that words don't go undefined.
The typical happy path for using Openshift is to use it's Source to Image (s2i) builder to take a git repo or dockerfile and turn it into a container running on Openshift. In general, this works just fine. The s2i builder detects which language your app is using and deploys that application appropriately, including starting the webserver. However, if your project needs more dependencies than Openshift expects, it's very difficult to add those dependencies to the container. For good reason, your user in the container is pretty neutered, which means installing a package is impossible. And you don't have access to the processes that are installing dependencies for your application, since those come from the default (probably CentOS or RHEL) base image that Openshift uses. So, what do you need if you need to install a package in your container? We'll explain!
In short, you'll need to:
- Copy the s2i builder that your application is using (based on it's language)
- Modify it as necessary for your application
- Create a new application in Openshift that uses your Dockerfile to create an image
- Create your "real" application in Openshift, using the image you just created as a base and the Source strategy
We'll go through it step by step!
Copy the Default S2I Builder
The best way to find the s2i builder your application will need is to reference the Openshift S2I documentation, find the language your application is in, and then in the first paragraph there should be a link to the git repo where the default s2i builder for that language lives.
Searching for 'Openshift $LANG S2I builder' also usually resulted in the first hit being the "official" s2i builder, but the git repos don't make that clear so going through the documentation should give you a little more certainty you found the right one.
I personally cloned the whole repo, then copied the relevant files into a subdirectory of my application so that everything I needed was packaged together. Later on, this will make things a little easier. If you'd rather separate your concerns though, you can also fork + download + modify the s2i builder and keep it separate from your application.
Here's what my application directory ended up looking like, where everything
under openshift-build
is copied from the Openshift ruby s2i repo.
. ├── app.rb ├── config.ru ├── Gemfile ├── helpers │ └── helpers.rb ├── openshift-build │ ├── cccp.yml │ ├── Dockerfile │ ├── root │ │ └── opt │ │ └── app-root │ │ └── etc │ │ ├── generate_container_user │ │ ├── puma.cfg │ │ └── scl_enable │ ├── s2i │ │ └── bin │ │ ├── assemble │ │ ├── run │ │ └── usage │ └── test │ ├── puma-test-app │ │ ├── app.rb │ │ ├── config.ru │ │ ├── Gemfile │ │ └── Gemfile.lock │ ├── rack-test-app │ │ ├── app.rb │ │ ├── config.ru │ │ ├── Gemfile │ │ └── Gemfile.lock │ └── run ├── public │ └── images │ └── approval.gif ├── README.md ├── spec │ └── kukla │ ├── helper_spec.rb │ └── kukla_spec.rb └── views └── index.erb
Key to Success: Make sure you include everything in the s2i
and
root
directories from the default s2i builder! These files are
necessary for building your first image later on.
Modify As Necessary
I was trying to trick Git into thinking I had a user on Openshift, so that I could clone some dependencies for my application. I also needed npm for Reasons. I looked at the nodejs s2i builder, which uses nss_wrapper to achieve the Git-trick, and just copy-pasted the important bits (apologies that the application isn't open source so I can just reference it):
< INSTALL_PKGS="rh-ruby24 rh-ruby24-ruby-devel rh-ruby24-rubygem-rake rh-ruby24-rubygem-bundler rh-nodejs6 rh-nodejs6-npm nss_wrapper" && \
---
> INSTALL_PKGS="rh-ruby24 rh-ruby24-ruby-devel rh-ruby24-rubygem-rake rh-ruby24-rubygem-bundler rh-nodejs6" && \
I also added this file from the nodejs builder to $APP_ROOT/openshift-build/root/opt/app-root/etc
so that my application could use nss_wrapper.
Create Application from Dockerfile
The next step is to create an image in the Openshift internal repository from our special S2I build. This will be the base image for our application, instead of the default Openshift builder image. I personally found this a little bit easier to do in the UI than on the CLI, but will explain both.
The key when you create this application is to use the Docker source
strategy using the Dockerfile (and all the other files!) you copied
from the default s2i builder. This is where it comes in handy to have
those files inside your application: if you point Openshift to where
those files are (spec.strategy.dockerStrategy.dockerfilePath
in the buildConfig), Openshift will use that for it's Docker build
strategy. As I describe in this blog post, it's difficult to create
a new application using your own buildConfig. I didn't feel like
messing with templates at this point, so I just:
- Created an application using the default image for ruby
- Edited the buildConfig
- Rebuilt the application.
Here's how it went down:
Note: I recommend naming this first application 'myapp-builder', or something along those lines.
In the CLI
- Run
oc new-app <appropriate configurations>
. As previously mentioned, Openshift will detect the language your application is in and use that default image. I highly recommend runningoc new-app --help
to see if there are any other options you'd like to use! - Run
oc edit bc/myapp-builder
and modify the strategy section of the buildConfig to have the following:
strategy: type: Docker dockerStrategy: dockerfilePath: openshift-build/Dockerfile env: - name: MYVAR value: my_value
- Close and save the buildconfig.
- Run
oc start-build myapp-builder
In the UI
Note: I believe the UI workflow only works if your file source is Github, not a Docker image
- Click 'Add to Project' in the header navigation
- Select the language (and version, etc.) your app is in
- Fill out the application details appropriately. There's nothing special you need to do on this page.
- Now that you've created your application, go to 'Builds -> Builds' in the left navigation
- Select the application you just created
- In the top right, select 'Actions -> Edit Yaml'
- Modify your buildConfig to use the Docker source strategy. The key changes here are under the
strategy
section. You should end up with something that looks like this:
strategy: type: Docker dockerStrategy: dockerfilePath: openshift-build/Dockerfile env: - name: MYVAR value: my_value
Then save the config 8. In the top-right corner, click 'Start Build'
And you're done! The resulting image will be published to the internal container registry with the label 'app-name:latest'.
Create Application from Source, Using Image
The final step, creating our application image from the image we just
created! The key when creating this application is to use the source
strategy with your image as the
source.strategy.sourceStrategy.name
value. This should be the
name of the application you just created. The steps for creating this
application are more or less the same as they were in the previous
step:
- Create an application using the default image in your language
- Edit the buildConfig
- Rebuild the application
You can follow the same steps enumerated above for the preferred interface, and here's roughly what your buildConfig should look like when you're finished:
strategy: sourceStrategy: env: - name: MYVAR value: my_value from: kind: ImageStreamTag name: myapp-builder:latest
That last line being the critical point! Make sure you're referencing your builder image.
And there you have it! Once you've rebuilt your application, you're all set.
Making Changes
If you make changes to your application, you only need to rebuild the application, not the application builder. However, if you make changes to your Dockerfile or any other part of the underlying image (say you need another package installed), then you will need to rebuild the app-builder image and your application.
Resources
Openshift is in an interesting place developmentally, where they have pretty good documentation, relatively mature technology, and awesome tech support on IRC, but almost no community documentation (think blog posts, stack overflow questions, etc.). I would often find myself reading a post from the Openshift blog that was published in 2013, and hope it was still relevant, or be looking through github issues and irc logs for answers. But, there were still a number of resources I found very helpful through this journey:
- First, I highly recommend asking any questions in #openshift (irc.freenode.net). Folks were kind, quick to respond, super helpful.
- The Openshift book is a great resource if you have the time. I would suggest only reading the first 1/3 of it or so to understand core concepts, then starting your own project.
- This doc is the best "Everything you need to know about Openshift builds" documentation
- This is a pretty good resource on the anatomy of an s2i builder, though definitely isn't required reading for this post.