tech blog

AppNexus is today’s most powerful, open, and customizable ad tech platform. Advertising’s largest and most innovative companies build their businesses on AppNexus.

How Test Environments Enabled Organizational Growth: Tech Talks @ AppNexus

| Comments

Watch this tech talk to learn how our DevOps team has built a scalable infrastructure (using Docker and other technologies) to automate the creation of scaled-down replicas of the production AppNexus platform and make them available company-wide as an on-demand service. Featuring AppNexus Engineering Team Lead Leevar Williams.

Watch this tech talk to learn how our DevOps team has built a scalable infrastructure (using Docker and other technologies) to automate the creation of scaled-down replicas of the production AppNexus platform and make them available company-wide as an on-demand service. Featuring AppNexus Engineering Team Lead Leevar Williams. Learn more about upcoming tech talks at http://meetup.com/TechTalks-AppNexus-NYC/

How to build, manage and scale an effective engineering organization: Tech Talks @ AppNexus ft. Stack Overflow CEO Joel Spolsky

| Comments

Think hiring the best engineers is tough? What about keeping engineers engaged once they are a part of your organization? Watch the latest installment of our Tech Talks series to hear how Joel Spolsky, founder and CEO of Stack Overflow, and Igor Shindel, CIO of AppNexus, think about these issues and discuss what it takes to build and manage a world-class engineering organization.

Think hiring the best engineers is tough? What about keeping engineers engaged once they are a part of your organization? Watch the video to hear how Joel Spolsky, founder and CEO of Stack Overflow, and Igor Shindel, CIO of AppNexus, think about these issues and discuss what it takes to build and manage a world-class engineering organization. This event is part of the Tech Talks @AppNexus series. Learn more at http://www.meetup.com/TechTalks-AppNexus-NYC/

Securing our Big Data Platform: Part 3

| Comments

Overview

As we’ve seen in our past couple of posts – Securing our Big Data Platform: Part 1 and Securing our Data Platform: Part 2 – we’ve been hard at work beefing up the security of our Big Data Platform. We had one other important piece of the puzzle to solve before we dotted our i’s on this security effort – Delegation.

Delegation is the process of letting a user act on the behalf of another user. Delegation allows user X to perform actions on behalf of user Y by granting the privileges of user Y upon user X. Thus, through delegation we are able to account for a user’s actions in our services.

Lessons learned in distributed systems disasters: Tech Talks @ AppNexus

| Comments

The latest installment of our ongoing Tech Talk @ AppNexus videos is now live on YouTube. Watch John Wittrock, Senior Software Engineer on the Real-Time Platform team, tell a variety of disaster stories that illustrate the challenges of and lessons learned from supporting our large-scaled distributed backend that serves millions of requests a second across thousands of servers.

We built a distributed system. By accident. Some of the decisions we made along the way turned out to be great, others not so much. Watch the video to learn about how we dug ourselves out of the desperate hole of multi-process and shared nothing, and how keeping our system topology flat has saved us a few ulcers. Featuring AppNexus Senior Software Engineer John Wittrock. Learn more about our tech talks at http://bit.ly/appnexustechtalks

XSS Vulnerability and Patch Explained

| Comments

Last week a security researcher, Randy Westergren, pointed out in a blog post a vulnerability from cross-site scripting (XSS) that was common across our industry and impacted AppNexus. We’ve since rolled out a patch to the immediate issue and encourage you to read Brian’s blogpost addressing this and publisher security in the broader context.

We also decided to prepare a more detailed technical view of the issues identified by Randy Westergren to share with our customers and partners. The identified vulnerability is different from malware threats where the main delivery mechanism is javascript called by a specific creative, an area we invest significant resources to prevent and fight. And when we look around the internet, we still see this XSS vulnerability through code owned by a number of other companies. We plan to reach out to these companies to notify them of this vulnerability and hopefully encourage a widespread fix across our industry.

Real Time Big Data: Tech Talks @ AppNexus

| Comments

Another in our “Tech Talks” series @ AppNexus.

Lambda architectures for Big Data sprung out of the concept that stream processing is fast but inaccurate. So you needed to have a fast but less accurate streaming path along with a slower, accurate batch path. With the advent of technologies like Ratsoda and Samza, this is no longer the case.

Lambda architectures for Big Data sprung out of the concept that stream processing is fast but inaccurate. So you needed to have a fast but less accurate streaming path along with a slower, accurate batch path. With the advent of technologies like Ratsoda and Samza, this is no longer the case. Featuring AppNexus Principal Engineer Riley Berton. Watch the video to find out how AppNexus does real-time streaming joins across our entire platform to the tune of 200+ billion rows per day with minimal latency on minimal hardware. Learn about upcoming AppNexus tech talks at http://www.meetup.com/TechTalks-AppNe…

Securing our Big Data Platform Part 2

| Comments

Overview

When last we saw our heroes, they were battling Hadoop authentication in DPaaS. But our intrepid knights were able to overcome that crafty foe and move on to the next obstacle: authorization. And that’s where our story continues…

Authorization is the control of access to resources or actions for an entity that’s requesting them. Implementing a proper authorization layer within our system is what will give our users the peace of mind that they are in control of their data & processes and are properly isolated from one another. As the operators of the system, we gain the same peace of mind and more. We get more control over the use of our finite resources. Better yet, we can take ourselves out of the critical path of our users’ needs — more automatic security means more self-service. That’s a big win-win.

Ad Viewability and Feature Selection for Big Data

| Comments

Data Science … Translating Theory into the Real World

In today’s post, I will unite the theory behind a machine learning technique with a real-world industry problem. I’ll describe one problem we needed to solve, how we solved it by building a new tool, and how you can also leverage it!

Hopefully after reading this, you’ll have a better idea how we use data at AppNexus. Or, maybe you can sympathize with me after encountering yet another memory issue, and this tool will help alleviate your pain!

Pyrobuf – A faster Python Protobuf library written in Cython

| Comments

Introduction

Pyrobuf is an alternative to Google’s Python Protobuf library that is written in Cython and that offers better performance (roughly 1.5-2x faster), Python 3 support, and simple serialization/deserialization to JSON and native Python dictionaries. Since Pyrobuf’s only installation requirements are Cython, Jinja2, and setuptools, its also much easier to install than Google’s library, and should work as a drop-in replacement. Pyrobuf parses the same .proto specs as the Google library and generates Python modules (in .so form).

Pyrobuf started as just one part of a larger Python “port” of a C library that we use for serializing and deserializing data between a variety of formats. We plan to open-source this library in the future, but in the mean time we saw such good results with the Protobuf portion of the library that it seemed worthwhile to release it on its own.

Software Transactional Memory is Simple

| Comments

Another installment of our ongoing series of tech talks has just hit the youtubes. Paul Khuong, previously of this fame, brings us “Software Transactional Memory is Simple” which takes us into the innards of the AppNexus real time platform and how we do configuration data updates in such a low latency environment.

AppNexus’ real-time adserving stack is built on non-blocking concurrency control, which is how we achieve sub 1% timeout rates. In practice, it’s easy to misuse these techniques, which can result in crashes or angry clients. In this tech talk AppNexus Principal Engineer Paul Khuong talks about software transactional memory, how to implement it (simply), and learn about the specialized non-blocking STM we use here at AppNexus. Sign up to our MeetUp group to stay up to date with upcoming tech talks: http://www.meetup.com/TechTalks-AppNexus-NYC/