Benchee on the web — BencheeWeb

About a month ago, I saw this Tweet sent out on the #MyElixirStatus hashtag. I clicked through to read Tobias’ blog. I pulled the code from hex (https://hex.pm/packages/benchee) and played around with it a bit.

I saved the tweet because I immediately thought of sites I’ve used in the past like jsperf, that let people write JavaScript and see which block of code performs better. I wanted to write that for Elixir for a couple reasons. First, I wanted to contribute to the Elixir community. Second, I wanted to create something in Elixir.

The problem was, I didn’t know enough Elixir at the time to pull this off. But I used this idea to drive my learning.

I created BencheeWeb (https://github.com/taylonr/benchee_web) that would take two code snippets and run them through Benchee.  My first thought was just to take the console log formatter and put it in a string. Unfortunately, it wasn’t the easiest thing to spit out in Phoenix. So instead of using that, I took the raw results and created an array of maps. Once that was done, I was able to display the results in a table. But that was all with a hardcoded function. That is, I wasn’t accepting user input. I was simply running a hardcoded Benchee statement and displaying the result on the web. Not exactly fun or useful.

My next challenge was to accept user input. I was stumped at exactly how to get that code into Benchee. As I was experimenting, I thought a macro might be the way to go. So I created a macro that would take the user string and create a Benchee.benchmark method call.

https://gist.github.com/taylonr/da5fd7c02959d48de9020811c57ff88a

The problem was, the function that was being evaluated was a string, which meant that no matter what function I tried to test, it always returned a median of less than 0.02 micro seconds.

The next step was to take the user’s string, combine it with the Benchee code as a string and evaluate it.

https://gist.github.com/taylonr/4e3c9084e8728771ce1b8b63bbf69d7f

This appeared to work, if the second snippet was bigger or more complex the results showed that. I was no longer getting sub-microsecond results. However, it seemed really slow.  I tweeted it out to Tobias and had him and someone else tell me it was from the code eval. Eval is slower than compiled code. I knew there was a Code.compile_string but I wasn’t really sure how to use it.

I tried to replace Code.eval_string on line 5 above with Code.compile_string and got no results back. It was because I misunderstood what I was doing. Compile_string expects a fully valid block of code, which meant I needed a module. I played around in iex trying things out with Code.compile_string. Then last night it hit me what I needed to do.

https://gist.github.com/taylonr/bdec86d515e926ad55cc97a31153a47c

Here, starting on line 5 I define an entire module. On lines 9 & 10, I interpolate the strings the user specified into the Benchee code.  Then on line 17, I compile that code and it’s now available to me. So I can run Sample.benchee on line 18 and get my results.  Now the results seem to be better that the eval version. I pushed it up and ran it on Heroku (https://blooming-sea-95724.herokuapp.com) and the numbers weren’t what I wanted, but I attribute at least some of that being that it’s on a free Heroku dyno. So future steps will be to try it out on a paid version.

BencheeWeb isn’t done or necessarily even production ready as there’s several features I’d like to add. Features such as configuring the time property on Benchee, it’s hardcoded to 3 right now. I also want to be able to create permalinks so that people can share their results. There are another 4 or 5 issues on the Github repo of things I’d like to do.

The repo is up on Github (https://github.com/taylonr/benchee_web) and if this is interesting to you, I’d love contributions, and would be happy to accept PRs.