About a month ago, I saw this Tweet sent out on the #MyElixirStatus hashtag. I clicked through to read Tobias’ blog. I pulled the code from hex (https://hex.pm/packages/benchee) and played around with it a bit.
The problem was, I didn’t know enough Elixir at the time to pull this off. But I used this idea to drive my learning.
I created BencheeWeb (https://github.com/taylonr/benchee_web) that would take two code snippets and run them through Benchee. My first thought was just to take the console log formatter and put it in a string. Unfortunately, it wasn’t the easiest thing to spit out in Phoenix. So instead of using that, I took the raw results and created an array of maps. Once that was done, I was able to display the results in a table. But that was all with a hardcoded function. That is, I wasn’t accepting user input. I was simply running a hardcoded Benchee statement and displaying the result on the web. Not exactly fun or useful.
My next challenge was to accept user input. I was stumped at exactly how to get that code into Benchee. As I was experimenting, I thought a macro might be the way to go. So I created a macro that would take the user string and create a Benchee.benchmark method call.
The problem was, the function that was being evaluated was a string, which meant that no matter what function I tried to test, it always returned a median of less than 0.02 micro seconds.
The next step was to take the user’s string, combine it with the Benchee code as a string and evaluate it.
This appeared to work, if the second snippet was bigger or more complex the results showed that. I was no longer getting sub-microsecond results. However, it seemed really slow. I tweeted it out to Tobias and had him and someone else tell me it was from the code eval. Eval is slower than compiled code. I knew there was a Code.compile_string but I wasn’t really sure how to use it.
I tried to replace Code.eval_string on line 5 above with Code.compile_string and got no results back. It was because I misunderstood what I was doing. Compile_string expects a fully valid block of code, which meant I needed a module. I played around in iex trying things out with Code.compile_string. Then last night it hit me what I needed to do.
Here, starting on line 5 I define an entire module. On lines 9 & 10, I interpolate the strings the user specified into the Benchee code. Then on line 17, I compile that code and it’s now available to me. So I can run Sample.benchee on line 18 and get my results. Now the results seem to be better that the eval version. I pushed it up and ran it on Heroku (https://blooming-sea-95724.herokuapp.com) and the numbers weren’t what I wanted, but I attribute at least some of that being that it’s on a free Heroku dyno. So future steps will be to try it out on a paid version.
BencheeWeb isn’t done or necessarily even production ready as there’s several features I’d like to add. Features such as configuring the time property on Benchee, it’s hardcoded to 3 right now. I also want to be able to create permalinks so that people can share their results. There are another 4 or 5 issues on the Github repo of things I’d like to do.
The repo is up on Github (https://github.com/taylonr/benchee_web) and if this is interesting to you, I’d love contributions, and would be happy to accept PRs.