Because builds can’t fail loudly enough or embarrass you in enough places, RunCodeRun now supports politely alerting you to your FAIL through Campfire. No need to install anything into your build, just tell RunCodeRun “Yes, ma’am, can I have another!”
First, log in and edit your account profile:
Then, click on the “Notifications” tab and fill in your Campfire credentials:
After that, you’ll see notifications like this interrupting your otherwise pleasant Campfire chatting:
Good luck avoiding the Build Bot! It can’t be bargained with, it can’t be reasoned with. It doesn’t feel pity, or remorse, or fear, and it absolutely will not stop. Ever. Until you go green.
We are pleased to announce that RunCodeRun is now accepting private builds. This means that you can now connect your private repositories on GitHub to RunCodeRun for your continuous integration needs. You can drop by our list of plans to see which best suits your situation.
We feel it important to note that RunCodeRun remains, as always, free for open source developers. If you are already an open source customer and want to upgrade, just sign in and head to the plans page and pick the option that works for you. You can keep all your open source projects active, now and forever, regardless of which plan you are on. And if you are just interested in getting started with an open source project? That’s easy: just sign up for an open source account. You can always upgrade later if you want to add private builds.
We look forward to seeing your projects clean and green!
Want to see what TDD is like in a functional language? In this blog I will convert the Bowling Game example, as TDD’ed by Uncle Bob, to a more functional approach in Clojure.
Uncle Bob Martin, well-known OO and agile guy, has used the Bowling Game to explore TDD in a variety of different languages. Most recently, he has been learning Clojure, and experiencing a disconnect from the idioms he would use in an OO language. I felt this same disconnect when I started using Clojure, and I think I have finally bent my brain into the shape needed to TDD functional code.
I have taken Uncle Bob’s example, and converted it to a more functional approach while keeping tests passing at each step. This blog summarizes the key elements of my thinking, but of course you can also follow it step by step in the commit history of the clojure-bowling repository.
A starting point
The initial commit of the project simply imports Uncle Bob’s code, and adds some supporting infrastructure (shell scripts to launch a REPL with all the code available, etc.)
In addition, I chose to use the development edge of Clojure and clojure-contrib to take advantage of the new syntax for the are macro (more on that later). The repository includes compatible versions of these JAR files so you don’t have to download anything else.
Data, not objects
In OO, it is customary to invest a lot in how objects get made. So, the code includes functions like new-game that creates a game. This starts to get ugly with the functions that make test data. For example, the roll-many function creates many identical rolls of a bowling ball, and the roll-list takes an arbitrary list of rolls and adds them to the game. Using these functions leads to some pretty hard-to-read tests. For example, here is one that tests scoring a game that starts with a spare, followed by all gutterballs:
Ouch. I can hear the parenthesis-haters sharpening their pitchforks in the background. But what happens if we think of the game as a simple sequence of data?
"one spare" 16 (concat [5 5 3] (repeat 0))
In this tabular-looking expression, the "one spare" names the case being tested, the 16 is the expected score, and the concat builds the list of rolls: 5, 5, 3, 0, 0, 0 … Note that we are taking advantage of Clojure’s lazy collections to ignore exactly how many zeroes are needed.
The tabular, FIT-like style of listing values is enabled by the are macro. There are tests for several different possible games, and it is easy to add more:
Scoring bowling is simple: “iterate over the frames of the game, adding the score for each.” Since we are using functional style, we will use recursion instead of iteration. Furthermore, since we are on the JVM, we will use Clojure’s explicit recur to trigger Tail-Call Optimization (TCO). Here is the original:
That works, but I think I hear the parenthesis-bashers giggling again. The score function is pretty tough to read, especially the loop with five positional arguments. What is more troubling is that this function relies on a supporting infrastructure of five other functions, most of which don’t make much sense alone. Here is score-spare:
(defn score-spare [rolls]
[2 (+ 10 (rolls 2))])
After puzzling a minute, it becomes clear: score-spare returns a pair:
the number of balls consumed by a spare frame
the score for this specific spare frame (ten plus the extra ball in the next frame).
Now I can hear the OO crowd sharpening their spears: “With objects it would be clear what was being returned here: the FrameStat bean or some such.”
Sequence transformation!
Instead of thinking about iterating over the rolls, think about transforming the rolls into a sequence of the thing you really want: the frames. So, for example:
The new sequence includes in each frame the balls needed to score that frame. Here is an implementation:
(defn frames
"Converts a sequence of rolls to a sequence of frames"
[rolls]
(when-let [rolls (seq rolls)]
(lazy-seq (cons (take (balls-to-score rolls) rolls)
(frames (drop (frame-advance rolls) rolls))))))
This function grabs (balls-to-score rolls) to make a frame, and then drops (frame-advance rolls) balls to advance to the next frame. Unlike the previous score-spare, these new helper functions cleanly express domain knowledge.
(defn balls-to-score
"How many balls contribute to this frame's score?"
[rolls]
(cond
(strike? rolls) 3
(spare? rolls) 3
:else 2))
Helper functions like strike and spare are also simple expressions of domain knowledge:
(defn strike? [rolls]
(= 10 (first rolls)))
Keeping score
With the toolkit of frames and its various helpers in place, it is simple to score an entire game. Just convert rolls to frames, score the frames, and sum the score. The function reads like a definition, once you are accustomed to the notions of map and reduce:
(defn score-game
"Score a bowling game, passed as a sequence of rolls."
[rolls]
(reduce + (map score-frame (take 10 (frames rolls)))))
Build a toolkit, not a tool
Because all of the functions that support score-game are individually valuable tools, it is easy to implement new domain concepts. If you wanted to get the individual frame scores of a game in progress:
(count (filter strike? (frames [10 0 10 0 10])))
-> 1 ; think about it
Parting shots
There is a lot going on here, but the single most important step is transforming sequences. Because the code now transforms sequences of balls to sequences of frames, it can be used to do all kinds of things other than just scoring the entire game. Moreover, the code is now shorter (in lines of code) and also more readable. Ditto the tests.
If you can think of some way to decompose a problem, TDD will help you build the individual pieces. But TDD does not help you choose the best decomposition. Functional decomposition is generally superior to object-oriented decomposition using mutable state.
Note that the use of laziness in the scoring is slightly gratuitous in this example, since bowling games are of a fixed size. However, laziness is a very useful part of solving real-world sized problems, and it is almost as easy to write as non-lazy code. So I tend to write laziness-preserving transformations even when I know (famous last words) that domain entities will never get large.
Resources
The functional programming chapter of my book Programming Clojure has more discussion of sequence transformations.
If you are new to Clojure, Mark Volkmann has written a long and thorough introductory article.
When should your Continuous Integration service notify you about the results of a build?
When we originally implemented email notifications for RunCodeRun, we thought to ourselves, “We get too much email as it is. Let’s not send an email after every build; let’s just send notifications when the build status changes. You know, when the build breaks, we’ll obviously want to know. And when when it goes back to green, we’ll want to know that as well.”
That seemed like a reasonable approach at the time. Then along came reality. You see, we do get too much email as it is. And as a result, sometimes when the build goes red, that status change notification might get lost in the noise. A few days go by – or even worse, a few weeks go by – and all the while, the dev team is committing away in WorksOnMyBox Land, none the wiser. Maybe the team introduced a dependency that happens to reside on the development machines, but they forgot to declare that dependency in the project. Maybe they unknowingly implemented a feature that only works on their development machine’s operating system or architecture.
If your deployment environment looks even the slightest bit different than your development environment (and who’s doesn’t?), these are things that you want to know about. And if you make another change to the code, and these problems still exist, you want to know.
With the changes we pushed out today, RunCodeRun will now remind you of your ill-fated builds after each build failure. And then when you fix the build, RunCodeRun will notify that you’re back to green.
If you’re not getting any build notifications at all, you can enable them in your account preferences. (You may also want to double check your spam folders, and be sure that you have bot@runcoderun.com on your email whitelist.)
RunCodeRun is known for free open source builds, but did you know it builds Java as well? RunCodeRun still assumes that you are using Rake instead of Ant, but thats easily remedied with a Rakefile that forks an Ant build (note that there are many resources for building Java with Rake or otherJRuby based tools as well). Since Ant isn’t available on RunCodeRun by default, you’ll need to include a couple of Ant jars in your source code.
Setup Ant
First off, add the following two jars to a “lib” or “vendor” directory: ant.jar and ant-launcher.jar. (I’m assuming you’re using a recent version of Ant.)
Then in your build.xml file, define (or find) the target that will build your code and run your tests. We’ll need this target name in the next step.
Setup Rake
Now, create a file called Rakefile in the root directory of your project. Put the following in your file:
# Ruby Rake file to allow automatic build testing on RunCodeRun.com
task :default => [:test]
task :test do
classpath = [File.join(".", "lib", "ant.jar"), File.join(".", "lib",
"ant-launcher.jar")].join(File::PATH_SEPARATOR)
exec "java -cp #{classpath} org.apache.tools.ant.Main -emacs all"
end
Note this snippet assumes that you’ve put the jar files in a lib directory and that you’re using an Ant target called all. The -emacs option to Ant doesn’t really have anything to do with Emacs, the editor. It just tells Ant not to print all the “[PROCESS NAME]” line prefixes it normally outputs. If you like those prefixes, just remove the option.
Lastly this assumes that Java is on the command line, and that your project runs on Java 6. Since a lot of our projects happen to use bits of JRuby or Clojure, we tend to stick with the latest updates of
Java 6. Your mileage may vary.
Testing from the Command Line
Now you’re ready to test your setup! I recommend creating a failing test in your project and calling ant all (or whatever your default build-and-test target was) to see the failure. Then immediately type the following from your command line: echo $?
This will output the return value of your ant program. It should be non-zero. If it’s zero, something’s wrong with your build, and Ant is not reporting failure correctly. (This would make all of your tests always green, even if they’re broken. Sounds like fun, but it isn’t.) Dig into your Ant build to find out why it’s returning the wrong value. (Hint: if you’re calling a <java fork="true"> tag anywhere, make sure you also have a failonerror=”true” attribute.)
If you have a non-zero value, though, then you’re ready for the next test. Try typing rake to have Rake call your Ant build. Again, you should see a failure and calling echo $? should give you a non-zero value. You’re almost ready!
Now go and fix your failing test and call running rake again. Type echo $?, and if you get zero, then you’re ready to go!
Set up your project on RunCodeRun, push to GitHub, and
check your build status. You’re building and testing your Java project on RunCodeRun! For a running example, check out env-js - a JavaScript browser environment that uses Rhino & Java for its tests on RunCodeRun.
It should be obvious from the fact that we built RunCodeRun that we at Relevance believe strongly in continuous integration. CI is more than just testing—it gives you something qualitatively different, over and above what good testing discipline gives you.
One example of that is what Chris Eppstein (one of the Haml and Sass team) did last week. He’s using RunCodeRun to test the Haml gem and plugin against multiple versions of Rails (including the 2.3 release candidate). Chris’ latest Haml build status page shows the full test suite being run four times before reporting success.
When we run your project’s tests under RunCodeRun, we set the RUN_CODE_RUN environment variable, so your tests can know where they’re running and do something special, if desired. Chris makes use of that in the first part of his Rakefile to set a different default task for RunCodeRun builds:
if ENV["RUN_CODE_RUN"] == "true"
task :default => :"test:rails_compatibility"
else
task :default => :test
end
The test:rails_compatibility task runs the entire test suite against several Rails versions, failing if the tests fail against any of those releases:
namespace :test do
desc "Test all supported versions of rails. This takes a while."
task :rails_compatibility do
`rm -rf test/rails`
puts "Checking out rails. Please wait."
`git clone git://github.com/rails/rails.git test/rails` rescue nil
begin
rails_versions.each do |version|
Dir.chdir "test/rails" do
`git checkout #{version}`
end
puts "Testing Rails #{version}"
Rake::Task['test'].reenable
Rake::Task['test'].execute
end
ensure
`rm -rf test/rails`
end
end
end
We’re absolutely delighted; RunCodeRun is perfect for this. As Chris’ comment points out, it’s a time-consuming process, so you don’t want to run all of these tests before every checkin; that would slow your development down. But you do want to learn relatively quickly if you’ve done something that will fail on older Rails versions. It’s an ideal workflow: run the ordinary test task before each checkin, and then have RunCodeRun test against all of the supported Rails versions after each checkin.
We hope to make this even easier in the future … we’ve already got a few ideas. For now, though, Chris’ approach works just fine.
We hope other plugin developers will follow Chris’ example. This makes it easier to support older Rails versions. By testing against the upcoming release candidate, you can ensure that you’ll be able to support the new version right out of the box. By testing against edge, you can get early warning of things that might break in the future.
Late last week, starting on Thursday, February 5th, we experienced an email outage due to some internal email notifications going haywire. This issue was fixed on Saturday. The outage meant that you may have missed build notifications during this period, so it’s possible your build was fixed or broken without your knowledge during this time. We apologize for any inconvenience this caused!
We’ve removed the offending notification and made some changes to our email system to prevent this from happening again. If you haven’t received notifications after Saturday that you think you should have, please let us know.
“My name is Stuart Halloway, and I believe it is OK to break the build.”
Several developers have recently asked me “How should developers who
break the build be punished?” With only that information, my answer is
easy: They shouldn’t.
Of course breaking the build is not a good thing, but let me make a
couple of hypothetical examples to demonstrate the point.
Example 1. Jane’s Birdhouses (JB)
JB is an ecommerce + social networking site for a brick-and-mortar
birdhouse store. Built with Test-Driven Development (TDD), JB has
three test suites:
a few hundred unit tests that execute in about 20 seconds.
a few dozen integration tests that execute in about 90 seconds.
a few dozen acceptance tests in Cucumber and Selenium that must be
run multiple times against different browser versions. As a result,
these “long tests” take about 30 minutes to execute.
Now, let’s consider how the JB build might break due to developer negligence:
A developer could commit code without bothering to run the unit
tests or integration tests locally.
A developer could commit code knowing that the tests were broken.
Both of these things very bad behavior, and should be condemned by
any agile team.
On rare occasions, extenuating circumstances might justify such
“negligent” commits, but documenting such corner cases is not my
purpose here. More interesting to me are the legitimate, respectable
commits that might nevertheless break the build.
For example, what if a developer commits code to JB that passes the
unit and integration tests, but breaks the long-running tests? That’s
not good, but it may be better than the alternative. If you insist
that developers run the “long tests” locally before committing to the
build box, then you cannot call yourself agile. Rapid feedback is
critical to agility, and taking several 30-minute breaks a day will
never meet that standard.
So you should not insist on perfect builds. What should you do if
a commit breaks the long tests? Stop what you are doing, figure out
why, and fix it. The problem will fall into one of three categories:
The problem could have been detected by a fast test, and that fast
test was an obvious thing to write. Solution: write the fast test, and
beat up the negligent dev.
The problem could have been detected by a fast test, but the needed
long test was not obvious until the long test found it. No
negligence here. Solution: write the fast test.
The problem could not have been detected by a fast test, or the cost of
writing the fast test is extremely high. This case sucks—but it does
justify running the long tests. Fix the problem, and know that similar
problems in the future will also break the build.
Another possibly legitimate build-breaker is uncovering a
hitherto-unknown dependency in an application’s setup. Similar rules
apply:
If the problem should have been obvious, then shame on the
developer. Example: adding a necessary Rubygem on a local box without
bothering to add it to the build (ie config.gems in a Rails project)
If the problem was more subtle, then just fix, learn, and move on.
Example: the latest version of support library Foo introduces a new
dependency on Bar. The authors of Foo did not document the dependency
clearly, and the dev didn’t notice because he already had Bar
installed.
Example 2. Jim’s Enterprise App (JEA)
JEA is a much bigger, more serious application than JB. It accesses
five separate databases, corresponding to five separate business
units. The tests are correspondingly more complex, and the test suite
takes four hours to run.
The JEA team is committed to quality. The original application didn’t
have unit tests, but the team has worked heroically to add them, and
code coverage now stands at 80.1%. Developers are docked on their
performance evaluations if they commit code that
breaks a test
drops coverage below 80%
How should we interpret this? Despite possible good intentions, the
JEA team fails agile at Square 1. First, the developers have to run a
four-hour test suite locally before committing. Otherwise, they risk
automatic censure for breaking the build.
Second, the coverage ain’t going to go much above 80%, ever. Think
about it: Bill writes a new module. When he runs the tests, coverage
goes up to 80.2%, but a test somewhere else in the app breaks. If Bill
simply deletes the other test, he meets both his objective
performance criteria: no broken tests, and coverage stays above 80%.
Talk about perverse incentives!
In my experience auditing code, I have seen far more codebases like
JEA than like JB. My guess is that most projects never partition their
tests into slow and fast, even when the test suite has grown so slow
that agile is a dimly-remembered dream. Moreover, should JEA even be a
single project? Probably not. Statistically, large projects suck, so
if you have any reasonable way to break your project into little ones,
you probably should.
Wrapping Up
Is breaking a build bad? Yes, sometimes. Usually, even. But context
matters, and teams that ignore context will never be agile.
You can now disable projects in RunCodeRun. “Disabling” basically means that the project and its history will be maintained, but future builds will not run. Any future pushes to that project will be ignored silently.
We will be adding some more smarts to the project listing soon, and disabled projects will not show up in the default view when those changes are in place. So if you are getting sick of dead projects cluttering up the view, disable them now and they will be filtered out soon.
GitHub recently rolled out a change to allow multiple post-receive urls per project. This allows you to integrate with as many services as you like from GitHub. Our instructions for getting started with Github integration have been updated to reflect this change.
More importantly, GitHub seems to have lost many of the pre-existing post-receive urls in the transition. If you’ve noticed that your project is no longer building on RunCodeRun, please go to your github project’s Admin Services tab and add “https://runcoderun.com/github” to your post-receive settings. Then fire off a test hook to get make sure things are working again. We had to reconfigure every project that Relevance has in RunCodeRun, so don’t be surprised if your project has not been pushing to RunCodeRun for the last couple days.
