A SECURE AND SCALABLE ACCESS CONTROL SOLUTION SIMPLIFIES AUTHORIZATION

NTWRK is a premier live streaming video commerce platform with approximately 2 million end users, tailored to serve the interests of sneaker aficionados, trading card collectors, and enthusiasts of various other niches. NTWRK prides itself on featuring a curated selection of exclusive sellers, offering unique products and unparalleled brand collaborations unavailable elsewhere.

NTWRK’s primary objective was to rapidly implement centralized access controls through a dependable solution. To achieve this, Steve High, Staff Engineer at NTWRK, opted for Cerbos as their authorization partner.

The collaboration resulted in a robust and scalable access control solution, enabling NTWRK to deliver top-tier user experience and concentrate on faster app iterations. 

We had a conversation with Steve to delve into the reasons behind NTWRK's choice of Cerbos, and to explore the remarkable outcomes of their partnership.

CHALLENGES

Q: Can you tell me a little bit about yourself and your role? 

Steve: I'm Steve High, a staff engineer at NTWRK. I've been with NTWRK for almost two and a half years. Before that I worked for Ardan Labs for a bit, and I previously worked for InVision and did agency work for brands like JP Morgan Chase and Wawa, so I've been doing this for 20+ years now. 

Q: If you don’t mind sharing, how many people are in the company as a whole and how large is the engineering organization?

Steve: There are around 150 people working at NTWRK. The engineering team consists of 32 people right now. Most of our developers float between our React front end and our Go backend services. There are probably around 5 of us right now that do work directly with Cerbos. Hopefully there will be more soon.

Q: Can you provide some figures for us to understand the scale of your operation?

Steve: In terms of end users, we have millions, I'm gonna say 2 million based on an unscientific database lookup. Most of those people are not currently affected by our Cerbos rollout. Right now, our Cerbos rollout only affects the admins of our application and our marketing partners. Next phase is to flip the switch and actually have every single user gated by some of our Cerbos policies. The focus of our work right now is to provide a multi tenant platform for our sellers. We believe if we attract great sellers, their fans will follow.

Q: How did you manage user permissions before you implemented Cerbos?

Steve: A lot of praying. We didn't really manage it. At this company at least, you were either an admin or an end user and that was it. There was really no need for an elaborate system at that time.

But now, we have been introducing all kinds of new roles to our administration and we needed not only role-based access control, which is kind of the easy part, but also the fine-grained tuning of permissions that the policies allow us to do. Which is where Cerbos came in. 

We have numerous edge cases regarding permissions, which often led to oversights when making changes, resulting in unintended side-effects. Somebody on the product team wants to change something and oh, we forgot to change it in 1of these 10 places. Cerbos, however, centralizes policies in one repository, simplifying management.

Q: Were you looking at any other authorization solutions?

Steve: We were examining AWS Cognito for a while. Cognito is a very good solution, but you have to position your infrastructure a certain way, otherwise it becomes a thing that you’re constantly fighting. And our infrastructure was not positioned that way. You need an unambiguous arrangement of tiers, and you need an edge or API gateway for Cognito to work nicely with minimal massaging. 

Whereas Cerbos, you can use it anywhere. I can use it 10 levels deep in repository code, if I wanted to. I wouldn't, because that's bad practice. But if I needed to, I could. And it wouldn't have any side effects, because the Cerbos connector is universal to my application and I can use it anywhere. We tried this with Cognito and it did not work. It was so painful. 

Q: What sold you on picking Cerbos as your chosen solution?

Steve: When I read through the Cerbos docs, I was blown away. It is really hard to make simple things well, and I think Cerbos is simple in a good way. It is very straightforward, there's no hidden “Gotchas!” that I've seen. It's so simple to onboard new people.

One of our big considerations was speed. We have strict latency tolerances. When it comes to Cerbos - you can call it a hundred times during a request and it doesn't matter. It’s incredibly fast. 

The Cerbos playground was also game changing for us. To be able to test your policies as you're writing them, isolated, that's a big deal. I really appreciate that you guys did that.

Another huge thing - if you're a decent developer, you can get Cerbos up and running in minutes, in a fairly straightforward way. There's not a ton of configuration and all the configuration there is, fits in one nice little file. 

Q: Was there any apprehension over using something that wasn't built specifically by you, for you, in terms of authorization?

Steve: No. I showed the team how Cerbos operates and got their buy-in easily.

When I worked at InVision, we had our own hand-rolled authorization system. And if we had Cerbos at that time, it would've been amazing, because we had to basically build our own distributed permission service. And there's just so many ways you can get something like that wrong. And we got it wrong myriad ways before getting it right.

So when Cerbos came along I thought, well, I can throw this entire concern across to a sidecar and get a yes or no result back. That's all I care about. 

SOLUTION

Q: How long did it take you to get started with Cerbos?

Steve: Let me give you the history of our application… When it was first built, it was built by an outside team of contractors who weren’t familiar with our business model. They built a monolith application in a single Go package (a feat that is equal parts amazing and terrifying), the impact of which we are still dealing with in some areas. And because the monolith was out there generating money for us at this point, we couldn’t just toss it out the window and do a rewrite. We had to, over time, lift and shift a whole bunch of stuff prior to implementing Cerbos.

So how I went about implementing Cerbos was, I went with a sidecar approach, mainly because I think that is the most performant, but also the safest. If we had a central service that you had to hit every time, I felt that was a little risky due to the latency tolerances that we have.

I was the only person working on the implementation and it took me a few months to go from nothing to where we're at right now. Which, I feel, is a really robust authorization system using Cerbos as the backbone. It was a natural progression. To use Cerbos in the code where it needs to be queried, now, it's very simple. I’m proud of how it came out, because it feels natural. I'm really happy with it. 

Had we not had to do the tech debt clean-up that I mentioned previously, we probably could have implemented much more quickly. Because Cerbos is plug and play. I mean, you can just plug it in and start using it.

Q: Can you walk me through a day in the life of using Cerbos?

Steve: Even though there's tons of documentation out there, it's just muscle memory for developers to ask the person who implemented something if any questions arise. I show them where the expression language is defined, go to use the playground to show the policies working in real time, and go over the API a little bit. It’s self-sustaining after that, because it's simple. It's very clean.

Other than that, the team and I mainly check policies periodically to make sure that they still line up with our business and product requirements. As far as the Cerbos code goes, that's set in stone now. I don't expect that to be touched anytime soon. 

Right now we're busy exploring how crazy we can get with the policies, and, so far, there's no limit. I've thrown some pretty gnarly attributes at Cerbos and it seems to be completely fine untangling that mess. Cerbos gets it right every time. And to me, when you have some really ugly edge case logic, the prospect of having to glue the two services together to do authorization is not appealing at all, but like I said, a million times already, Cerbos makes that a non-issue. Cerbos’ audit logging is great for debugging policy failures.

Q: What are some of the most prominent benefits of your collaboration with Cerbos?

Steve: It's the peace of mind knowing that I don't have to write janky code to do the same thing. I know that I don't have to write a bunch of database queries, hit MySQL for this and hit Redis for that. Just to be able to say yes or no, this person can do this thing. So to me, that's the main benefit that I get, is knowing that I just don't have to write ugly one-off code anymore. And honestly, that should be your tagline. You don't have to write ugly one-off code anymore.

The fact that we have one repo of policies with Cerbos, and knowing that we don't have to hunt for the same piece of code in five different repos, has made this collaboration well worth it for us.

As I mentioned previously, with the latency tolerances that we have, anything over a few milliseconds can be slow for us, depending on the context. If you imagine 10,000 people trying to buy the exact same instance of a physical product on our platform at the same time, that's what we have to deal with. Using Cerbos as a sidecar, we’ve been able to get permissions-checking latency down to microseconds, it’s ridiculously fast. In turn, NTWRK is able to provide a great user experience to our customers, both internal and external. 

Throwing the entire concern of authorization across to Cerbos really did increase my velocity, which in turn increases velocity downstream from me. It has allowed the team to deliver top-tier user experience and concentrate on faster app iterations.

Right now, only our internal employees can access our admin portal. But we're basically opening that whole thing up to the outside world. As part of that, we're using Cerbos to slice and dice permissions, to help us generate tokens. For example, we use a SaaS that essentially is a managed Elasticsearch. You can index all your data with them and create a token that says, okay, the person who's using this token can see these things, but not these things. We’ll use Cerbos to make that determination. We clear that with Cerbos first, before going through the expensive process of generating an ad-hoc API token. Cerbos is helping prevent us from making expensive decisions code-wise before we have to make them. That's probably an unsung benefit. 

RESULTS

Q: What would have happened if NTWRK had not deployed Cerbos?

Steve: We would probably have had to go with some kind of heavy handed, federated authorization system like SAML, Cognito or Auth0. That would probably have caused us to have to restructure quite a few things again. Which I don't wanna do, because I like the way our infrastructure is taking shape. I'm of the philosophy that you shouldn't have to change your application to suit tooling.

With Cerbos, we can just do whatever we need to do and then we have this little global Cerbos instance sitting there that we can query at any time, and it just works.

I guess if we couldn't use Cerbos, another alternative is I would have written some kind of abstraction that acted like Cerbos, honestly. Mainly, because I'm just really happy with the ease of it. I used to stress about authorization, and Cerbos has removed an entire stress point for me around distributing permissions in general. And like I said, if I had this five years ago, I'd probably be much happier. 

Q: Has anything exceeded your expectations since working with Cerbos?

Steve: Your support is awesome. Every time I get stuck on something, the Cerbos team is just a slack message away. If the Cerbos team doesn’t help me solve my question, they throw ideas at me, and each time I get unblocked. And you guys are working. The team is very punctual and gets back to me within an hour or so, which, you know, that's pretty amazing. 

I doubt I would have the same experience if I was having issues with Cognito. I would probably have to wait weeks for an appointment with the AWS professional who is going to say something akin to “just use this AWS-exclusive tool”, whereas you guys actually help me solve my problem in my problem space, so I appreciate that. 

Q: If there’s one word you could use to describe your experience with Cerbos, what would it be and why?

Steve: I would probably use simple. Simple in terms of implementation, and the tooling is very developer friendly. I would definitely make that the top tier thing that comes to mind when you think of Cerbos, because that was my experience pretty much, especially local development. I just brew install Cerbos and run it, and then point at it. And when I'm developing locally, I don't even think about it. It's just back there running in the background.

Q: If you were to recommend Cerbos to someone, what would you tell them?

Steve: Cerbos looks at all your roles, permissions, and everything about your user and says yes or no, can you do this thing with this thing? It's that simple. You create all the rules in the form of policies and Cerbos just compiles them all into this in-memory thing that you can then query and you can ask it the most complex questions and it will give you the right answer every time.

For people who've been through the pain of authorization, I would just say, all those weird edge case-y decisions - push it to Cerbos, because it just works. 

For any kind of startup, or anybody that's tried to do any kind of authorization, honestly, I would recommend it just for the sake of it's really easy to set up and run. If you're a decent developer, you can get Cerbos up and running in minutes, in a fairly straightforward way. That's a huge thing. 

WHAT’S NEXT?

The team at NTWRK looks forward to further scaling their app with the support of Cerbos, and having every single one of their users gated by Cerbos policies in the near future.

You can read the full case study with NTWRK here.