If your company is anything like AppNexus, your engineers are heads down solving your tough business problems. There might not be as much time as they’d like to work on a really interesting problem that isn’t urgent or that kooky project that will never be on the roadmap, but will make life at your company better. Here at AppNexus, we started hosting hackathons to solve this problem. So far, we’ve done two and our engineers have really enjoyed both.
In this post, I’m going to explain why it’s worth it to have a hackathon, what went behind hosting a great hackathon, and all the projects our engineers developed. I’m going to throw in my disclaimer here: I am not an engineer. I work with the engineering teams at AppNexus on a daily basis, but have only committed code once and it did not go so well.
Why have a hackathon?
The engineers here are usually collaborative, but the hacktahon was a fun time for them to get really competitive with each other. It also gave the engineers a chance to build some really cool projects that aren’t driven by the product roadmap or are a dream product.
Top Five Hackathon Best Practices:
Balancing having a fun hackathon and having a productive hackathon is tough. Below is a list of a few things we’ve learned that might help you have a successful company hackathon.
5. Have your hackthaon on a weekday: We got just as much, if not more work done during our hackathon, so it’s worth it to give up a day to a hackathon.
4. Give hackers some notice: Hackers need to form teams, develop ideas, and plan for the hackathon, so give at least a week or two of notice. Hackers should not be working on projects before the actual hackathon. Our teams had about 4-5 people on them.
3. Celebrate your hackers: For AppNexus, it was a no brainer to have a happy hour after the hackathon since we all enjoy hanging out with each other.
2. Demo! And make it a company-wide event:While not everyone in the company participated in building technology, they should get to participate somewhat. Get the whole company to watch the demos, vote on a winner, etcetera. (We gave participants a full vote and non-participants got ½ a vote, which led to some popular vote drama, but in the end we decided hackers deserved more voting power than non-hackers).
1. Put projects into production: Put the projects that should be productionalized into production or else you are wasting your engineers’ time. We weren’t as strict about this in the first hackathon, but in the second one we are devoting production cycles to building in the features that would benefit AppNexus. Engineers were much happier to find out their work would be used in the product (if appropriate).
What we’ll try next time: In our next hackathon we are tossing around the idea of pairing up our sales team with the engineering team and having the sales person help pitch the product during the demos.
Enough logistics. What did we build?
Our projects varied from browser plugins, a take over of our TV system, to solutions for improving our data scale solutions. Our projects are below, starting with the winner, then in no order particular order after that.
Even Sexier APIs: Not everyone at our company is great at using our APIs even though they are very sexy, and while you can use our UI to interact with our APIs, the UI web pages don’t map 1:1 with our API services. A few members of our API team, along with some other folks, built a way for users to interact directly with the API using HTML web pages, similar to filling out forms. Now when you want to make one specific API call, you no longer have to use curl and create complicated JSON strings by hand. This project actually won even though it was one of the simpler projects because it will make a lot of people’s day-to-day life better.
Data Scale: We have multiple Netezza’s (data warehouses), but as our company scales and we need to process 12 billion ads calls a day, we can’t just load raw logs directly into Netezza. In order to reduce the amount of load on our Netezzas a few members of our Data Team built a hadoop cluster where logs will be joined and processed before being loaded into our Netezas. Now our DMF (Data Management Framework) triggers a hadoop job, which joins two of our raw ad-server logs and generates an intermediate aggregation, freeing up space in Netezza.
Memory Accounting System: We incorporated an in-house memory accounting subsystem into a majority of the code that supports our ad server. The accounting system allows us to track memory usage by object type in order to better detect memory leaks and strategize performance optimization. Additionally, a GHT wrapper was implemented around our in house version of a hash table, an_table, that presented over-all reduced memory use and allows us to account for memory used by our hash tables.
AppNexus Universe: In our system, it’s difficult to get a broad view of your account, so a few engineers created a visual representation of all objects created in an account. The relational object viewer is a tool that enables users to visually navigate through their object hierarchies in the database. You can traverse the contents of an entire account, from the root member level all the way down to the most granular levels like profiles and pixels.
Publisher Monetization: Publishers represented by Networks do not have an easy way to see how their inventory is being monetized. Tagification, a browser plugin built by a few of our execs, allows for a high level of transparency for publishers. It detects any AppNexus ad calls on the page and provides a quick interface for a publisher to see all the necessary information including who the buyer is, price floors set by the network, and last but not least a small chart of the bid-landscape with all the non-winning bids. This is more information than most mid volume publishers receive.
Data Visualization: While a lot of AppNexians would agree with the fact that massive excel spreadsheets of data that you can manipulate any which way you want are the coolest, a lot of people like data to processed and put into a visual representation for them. A few of our data nerds joined forces and pulled data from our system, from the US census data, and other sources to visualize ad response data in a state map in a web interface.
Budget Data Streamline: Our real time bidding system needs up to date information to make informed decisions, so our budgeting system is updated every 15 minutes. This is a massive update (about 200 MB) of budget data on all objects where budget can be set: insertion orders, line items, campaigns and creative. We reduced the amount of data that needed to be updated by only tracking budget on objects that actually had budgets set instead of all objects where a budget could be set. This led to a 95% reduction (200 MB to 5 MB) in the amount of data that needed to be transferred.
Team Data Market Place: A few of our more technical client-facing employees got together and created the first AppNexus Data Marketplace. The data might be completely fake and we may not have a way to actually buy or sell the data, but the graphs were pretty and it almost won the popular vote since most of the business side of the company voted for it.
TV System Hack: I’ve been trying to demonstrate that each project solved a business problem we have, but this one does not quite fit that format—which is great! The only rule for the hackathon was that it had to be an AppNexus related project, not that it had to solve a business problem. But maybe I’m wrong; and it is a business problem that employees are not greeted by their favorite song and music video upon entering the office.
Since the Google TVs mounted throughout our office run a browser (Chrome), we decided that instead of just pointing each TV at a static page, we would give all TVs an arbitrary ID. Using this ID, we could then build a webapp (one part UI, one part server) to let us control the content of the TV from our computers, phones, etc. They also hacked our security system and set up the TVs in the lobby to welcome the person and play a song based on who scanned in.