Building Harbour's API v2: Authentication and Authorization (Part 2)

Part 1 of this series explored the foundational decisions that shaped Harbour's API v2. Now, let's dive into one of the most critical aspects of API design: authentication and authorization. This piece wasn't just about securing our API—it was about creating a flexible system that could grow with our product while maintaining security and usability.

Series Overview:

• Part 1: Foundations and Strategic Decisions
• Part 2: Authentication and Authorization (you are here)
• Part 3: Resource Design and Modeling
• Part 4: Error Handling and Validation
• Part 5: Performance and Scalability
• Part 6: Developer Experience
• Part 7: Operational Excellence

The Challenge: Beyond Simple Authentication

When we started redesigning our authentication system, we faced several challenges:

1. Multiple Client Types: We needed to support various integration patterns, from simple script-based integrations to complex service-to-service communications.
2. Fine-grained Permissions: Our evolving product introduced concepts like workspaces and groups, requiring more sophisticated access control.
3. Performance at Scale: The authentication system needed to handle thousands of requests per second without becoming a bottleneck.

Design Considerations

Rather than implementing a one-size-fits-all solution, we adopted a multi-layered approach that could adapt to different use cases while maintaining consistent security standards.

Authentication Layers

We implemented three distinct authentication methods:

API Keys: For all external integrations

• Simple, predictable authentication method
• Single responsibility: identify and authenticate the customer
• Support for multiple keys per account (e.g., development and production)
• Deliberately streamlined to reduce integration complexity

OAuth 2.0: For service-to-service communication

• Standard-compliant implementation
• Support for refresh tokens Automatic token rotation

Session-based Authentication: For our web application

• Secure cookie handling CSRF protection
• Integration with SSO providers

Authorization with ReBAC

Our move to Relationship-Based Access Control (ReBAC) was the most significant change. This decision was driven by the limitations we encountered with traditional Role-Based Access Control (RBAC).

Traditional RBAC approach:

{
    "user": "user123",
    "role": "admin",
    "organization": "org456"
}        

Our ReBAC model:

{
    "user": "user123",
    "relationships": [
        {
            "object": "org456",
            "relation": "admin"
        },
       {
            "object": "project789",
            "relation": "viewer"
       }
    ]
}        

ReBAC allows us to:

• Model complex organizational hierarchies
• Support granular permissions at any level
• Enable contextual access decisions
• Scale permissions without exponential complexity

Implementation Details

API Key Authentication

For API keys, we implemented a hash-based system with prefix support for easy identification:

def verify_api_key(key):
     prefix, env, token = parse_key(key)
     if not is_valid_prefix(prefix) or not is_valid_env(env):
          raise InvalidKeyError()
     hashed_token = hash_token(token)
     key_data = db.find_key(hashed_token)
     if not key_data or not key_data.is_active:
          raise InvalidKeyError()
     return key_data        

OAuth Implementation

Our OAuth 2.0 implementation focuses on security and developer experience:

@app.route('/oauth/token', methods=['POST'])
def token():
    client = authenticate_client(request)
    grant_type = request.form.get('grant_type')
    if grant_type == 'authorization_code':
         return handle_authorization_code(request, client)
    elif grant_type == 'refresh_token':
         return handle_refresh_token(request, client)
    raise UnsupportedGrantType()        

ReBAC Authorization

We used Google's Zanzibar paper as inspiration for our ReBAC implementation:

class RelationshipService:
     def check_permission(self, subject, action, object):

         # Direct relationships
         direct = self.get_direct_relationships(subject, object)
         if self.satisfies_permission(direct, action):
             return True # Inherited relationships (e.g., team membership)
         inherited = self.get_inherited_relationships(subject, object)
         return self.satisfies_permission(inherited, action)        

Lessons Learned

Our journey in building and implementing Harbour's authentication system has taught us valuable lessons that can benefit others in similar endeavors.

Embrace Simplicity

One of our most impactful realizations was that simpler solutions often lead to better outcomes:

• API Keys Are Often Enough: For most B2B integrations, API keys provide the perfect balance of security and usability. We initially considered more complex schemes but found they added unnecessary complexity.
• Resist Premature Optimization: Start with proven, straightforward patterns before introducing complexity. Many authentication problems can be solved without sophisticated mechanisms.
• Focus on Real Problems: Build solutions for actual customer needs rather than theoretical edge cases. Our customers consistently prefer simple, reliable authentication over feature-rich but complex systems.

Design for Growth

While keeping things simple, we also learned to build with the future in mind:

• Establish Strong Foundations: Invest time in core authentication patterns that can evolve without breaking changes. Our API key format and validation logic were designed to accommodate future enhancements.
• Consistent Patterns Scale Better: Standardized approaches to authentication and authorization make the system easier to maintain and extend. This consistency has been crucial as our product grows.
• Plan for Evolution: Design your system so it can grow without forcing disruptive changes. We've been able to add features like environment-specific keys without breaking existing integrations.

Documentation Drives Adoption

Clear documentation proved to be as important as the technical implementation:

• Comprehensive Getting Started Guides: Provide step-by-step guides that take developers from zero to their first authenticated request. Include code examples in multiple languages.
• Real-World Examples: Share concrete examples that solve common use cases. Our documentation includes examples of different programming languages and frameworks.
• Explain Design Decisions: Help developers understand why certain choices were made. This builds trust and helps them make better decisions when integrating.
• Keep Everything in Sync: Ensure documentation stays current with the API. Outdated examples or incorrect information quickly erodes trust.

Think About Developer Experience

The success of an authentication system largely depends on how developers interact with it:

• Clear Error Messages: Invest in detailed, actionable error messages. We've seen a significant reduction in support tickets by providing clear guidance when things go wrong.
• Predictable Behavior: Authentication should work consistently across all endpoints. Developers should never be surprised by how authentication behaves.
• Test the Integration Flow: Regularly go through the integration process from a customer's perspective. This helps identify friction points and opportunities for improvement.
• Simple Solutions for Common Issues: Make it easy to solve frequent problems like key rotation or adding new environment keys. The easier these tasks are, the more likely they'll be done correctly.

Looking Ahead

While our current authentication system serves our needs well, we're planning several enhancements to improve its capabilities:

Performance Optimizations

• Permission Check Caching: Implementing a caching layer for permission checks to reduce database load and improve response times
• Enhanced Query Optimization: Further optimizing our permission queries for complex organizational structures
• Request Batching: Adding support for batch operations to reduce API calls

Audit and Compliance

• Comprehensive Activity Logs: Adding detailed tracking of authentication events and permission changes
• Access History: Providing customers visibility into how their API keys are being used
• Usage Analytics: Offering insights into API usage patterns and potential security concerns

Security Enhancements

• Key Rotation Automation: Making it easier for customers to regularly rotate their API keys
• Enhanced Rate Limiting: Implementing more granular rate limiting based on customer needs
• Suspicious Activity Detection: Adding automated monitoring for unusual authentication patterns

Stay tuned for Part 3 of this series, where we'll explore our approach to resource design and modeling, including how we handle complex business relationships and query parameters.