Change Jenkins Jobs over HTTP using Java

In our existing build infrastructure which is based on Jenkins, we do almost all of the job creation and manipulation issues not manually, but instead use a Java tool which is capable of creating, manipulating and deleting Jenkins jobs. The basic idea behind this tool is to have configuration templates (XML files), process them with some Apache Velocity magic in order to have the values in it we need (e.g. Subversion URLs), and thus turn them into ready-to-use config.xml files.

In almost all cases the actual manipulation of the jobs is done using the Jenkins CLI (Command Line Interface). A general introduction into Jenkins CLI is found here. The idea behind the CLI is to simply issue commands from a command line and thus manipulate an existing CI system. The CLI is written in Java so it’s also possible to integrate it into other Java tools.

However, at the time of writing this article (May 2012) it was not possible to alter an existing job using the CLI. I’m not an expert of what’s going on behind the scenes at Jenkins, so I can’t give any explanation here. In order to have a workaround we meanwhile changed the config.xml files directly on the file system and then reloaded Jenkins’ configuration. But the disadvantage of this approach is that one needs to be on the Jenkins build server; no remote execution is possible of course.

Jenkins’ documentation clearly states that using the URL of an existing Jenkins job like http://localhost:8080/job/Evaluation_1_TRUNK/config.xml and issue an HTTP POST request against that URL providing a new file config.xml, will alter the job. Doesn’t sound too difficult and it actually isn’t. However, our Jenkins is secured and therefore I had to provide sufficient privileges in order to succeed with my POST request. I found various information but it took me quite a while to get everything done. So this post is about showing the result of altering an existing Jenkins job using the HTTP interface.

After some initial evaluations where I tested Java’s java.net package and Apache HTTP Components, I decided to use the Apache solution since it seemed to be the more sophisticated implementation. (And it’s always a good idea once a Java problem arises to look at what the Apache Foundation has already implemented.) The second reason for using Apache HC was that the Jenkins Wiki already provides a good example of how to authenticate against an HTTP server here. You will find some of the code there also listed here in this post. Hope you don’t blame me for simply copying, but there are a few things which make my solution different:

  • I had to use a HTTP POST request, not GET.
  • I wanted to have Jenkins access with and without user credentials.
  • I wanted to have a different response handling.

And since I wanted to provide a complete solution here, I just reused the code from the Jenkins Wiki site.

I will give a step by step introduction into the components needed for establishing an HTTP connection and will show in the end how everything glues together. Sometimes methods will be shown, sometimes only smaller code snippets. It’s up to the reader to create a ready-to-use Java class out of it. But I can promise, it’s quite easy since most of the needed code is shown.

Initializing the HttpClient
Let’s start with the most important component, the HTTP client. We will have to use the class

import org.apache.http.impl.client.DefaultHttpClient;

.

    DefaultHttpClient createHttpClient(String ciUser, String ciPasswd) {

        DefaultHttpClient httpClient = null;

        httpClient = new DefaultHttpClient();
        LOG.debug("New HTTP client created: " + httpClient.toString());

        if (ciUser != null && ciPasswd != null) {

            httpClient.addRequestInterceptor(new PreemptiveAuthInterceptor(), 0);
            httpClient.getCredentialsProvider().setCredentials(new AuthScope(AuthScope.ANY_HOST, AuthScope.ANY_PORT),
                    new UsernamePasswordCredentials(ciUser, ciPasswd));

            LOG.debug("HTTP client with credentials configured for user '" + ciUser + "'.");
        } else
            LOG.debug("No user information given. Will use HTTP client without credentials configured.");

        return httpClient;
    }

The method’s signature (line 1) allows to hand over the user credentials. In this case, the client has to be provided with appropriate authentication stuff (line 10-12). The class PreemptiveAuthInterceptor in line 10 will be discussed later. If no user data is given, we simply initialize the client (line 5); no further configuration is necessary.

Initializing the HttpContext
The HTTP context is needed later in order to actually execute the HTTP request. It also contains authentication issues. The class to be used is

import org.apache.http.protocol.BasicHttpContext;

The method which creates the context looks like this:

    
BasicHttpContext createHttpContext() {

        BasicHttpContext httpContext = null;

        httpContext = new BasicHttpContext();
        httpContext.setAttribute("preemptive-auth", new BasicScheme());
        LOG.debug("New HttpContext created: " + httpContext.toString());

        return httpContext;
    }

The preemptive authentication scheme is set to a basic scheme which will allow it later to set credentials based on a user’s name and password.

Initializing the the HttpRequestInterceptor
We also need a request interceptor based on the interface

import org.apache.http.HttpRequestInterceptor;

Its only method is process(HttpRequest request, HttpContext context) It will be executed on the client side before the request is send to the server, and on the server’s side it is executed before the message body is evaluated. It looks like this:

    
static class PreemptiveAuthInterceptor implements HttpRequestInterceptor {

        @Override
        public void process(@SuppressWarnings("unused") HttpRequest request, HttpContext context) throws HttpException,
        IOException {

            AuthState authState = null;
            AuthScheme authScheme = null;
            CredentialsProvider credsProvider = null;
            HttpHost targetHost = null;
            Credentials creds = null;

            authState = (AuthState) context.getAttribute(ClientContext.TARGET_AUTH_STATE);

            if (authState.getAuthScheme() == null) {

                authScheme = (AuthScheme) context.getAttribute("preemptive-auth");
                credsProvider = (CredentialsProvider) context.getAttribute(ClientContext.CREDS_PROVIDER);
                targetHost = (HttpHost) context.getAttribute(ExecutionContext.HTTP_TARGET_HOST);

                if (authScheme != null) {

                    creds = credsProvider.getCredentials(new AuthScope(targetHost.getHostName(), targetHost.getPort()));

                    if (creds == null)
                        throw new HttpException("No credentials for preemptive authentication");

                    LOG.debug("Got credentials from host " + targetHost.getHostName() + ": " + creds);
                    authState.setAuthScheme(authScheme);
                    authState.setCredentials(creds);
                }
            }

        }
    }

What is basically done here is is to retrieve the authentication state from the HTTP context (line 13) and fill it with the authentication scheme and the credentials (lines 29+30). See the code snippet further above which creates the HTTP client. In line 10 we assign this interceptor to the client.

Altering the Jenkins Job
Now we have prepared all components we need for altering an existing job. The next code snippet shows only an excerpt from my slightly more complex code, but it will show all necessary issues. I also did not list exception handling in this snippet.

StatusLine statusLine = null;
BasicHttpContext httpContext = null;
DefaultHttpClient httpClient = null;
HttpPost httpPost = null;
HttpEntity requestEntity = null;
HttpEntity resultEntity = null;
int statusCode = -1;
HttpResponse response = null;

  httpClient = createHttpClient("someuser", "somepassword");
  httpContext = createHttpContext();
  httpPost = new HttpPost("http://url.to.jenkins:8080/job/YourJobName/config.xml");
  requestEntity = new StringEntity(FileUtils.readFileToString("/path/to/new/config.xml"), "application/xml", "UTF-8");
  httpPost.setEntity(requestEntity);

  response = httpClient.execute(httpPost, httpContext);
  resultEntity = response.getEntity();
  statusLine = response.getStatusLine();
  statusCode = statusLine.getStatusCode();

  if (statusCode != HTTP_OK)
    LOG.error("Failed altering job"");
  else
    LOG.info("Successfully altered configuration.");

  EntityUtils.consume(resultEntity);
  • Line 10 creates the HTTP client with the method shown above. You have to give the user name and its password. The user must have sufficient rights on Jenkins to alter existing jobs.
  • Line 11 creates the HTTP context. The method is also shown above.
  • Line 12 creates the POST object. Here you have to give the complete URL pointing to the file config.xml of an existing job to be altered.
  • By creating the request entity in line 13 we initialize the ready-to-use file config.xml. In this case, we use the textual content of the file.
  • Line 14 assigns this file to the POST object.
  • The actual execution of the request takes place in line 16.
  • Lines 17 to 24 simply evaluate the result by using the returned StatusLine object and the returned status code.
  • Line 26 cleans up the result entity.

Conclusion
Using simple user name and password authentication might seem not secure enough. However, I think in most cases when it’s about a CI system like Jenkins, this should be sufficient. At least in our case it is.

Looking at the solution described above will make it obvious, that this is an ideal basis for iterating over a set of jobs and alter their configuration. Actually this is exactly what I do. However it’s a very special solution which only suites our particular infrastructure, so I will (and must) not show the detailed solution here. But it should be possible for everyone familiar with Java to build a framework around it.

I’m not an HTTP expert. So some information here might not be deep enough. But it should at least be sufficient to create a Java based solution which will do the job. Feel free to ask or comment any time!

Subscribe to new posts

If you're interested in new posts and updates, just subscribe here.

, , , ,

No comments yet.

Leave a Reply

* Copy This Password *

* Type Or Paste Password Here *