Building the Auth Foundation for Scrappy

Scrappy is my ongoing project for building a personal, AI-enabled document and chat system. The larger goal is to create a system that can ingest files, process documents, generate embeddings, support retrieval-augmented generation, and give users a clean interface for working with their own knowledge.

But before any of that, the system needs to know who is using it.

That is why the first major backend service I built was a dedicated authentication API.

In a production application, I would usually prefer to lean on a trusted identity provider like Google, Auth0, Clerk, or Cognito rather than rolling custom authentication from scratch. Authentication is security-sensitive, and managed providers are often the better default choice.

For Scrappy, though, I wanted this part of the project to demonstrate the underlying system design. Rather than hiding authentication behind a third-party provider, I chose to build a custom auth service that handles accounts, login, token validation, and logout directly.

That decision also fits the direction of the project. Scrappy may eventually include multiple frontends, background workers, internal tools, admin panels, and downstream APIs. Each of those services should be able to trust one shared authentication layer instead of recreating user management, token validation, or session logic on its own.

The auth API gives Scrappy a single source of truth for users, credentials, roles, sessions, and token validation.

Why I separated authentication from the main app

A lot of small projects start with authentication directly inside the main backend. That works at first, but it becomes harder to maintain once the system grows.

Scrappy is not just a single-page app talking to one API. The architecture is already moving toward multiple services:

  1. A frontend application
  2. A FastAPI backend
  3. A document ingestion pipeline
  4. Celery workers for background processing
  5. PostgreSQL for application data
  6. Redis for caching and job infrastructure
  7. MinIO or object storage for uploaded files
  8. Future services that may need to trust the same user identity

If every service manages authentication differently, the system becomes fragile. A centralized auth API lets the rest of the platform ask one question consistently:

“Is this request coming from a valid user, and what is that user allowed to do?”

What the auth API handles

The first version of the auth API focuses on the core account lifecycle.

It authenticates users with credentials, issues JWT access tokens, validates authenticated sessions, supports logout and token invalidation, and provides a foundation for managing users and roles.

The goal is not to overbuild a full enterprise identity provider. The goal is to create a clean, understandable, reusable authentication layer that can support the rest of Scrappy as the project grows.

At a high level, the service is responsible for:

  • Creating and managing user accounts
  • Authenticating users through login
  • Issuing signed JWT access tokens
  • Verifying tokens for protected routes
  • Supporting logout through token invalidation
  • Managing role-based access patterns
  • Providing downstream apps with a consistent identity layer

That gives the rest of the system a cleaner contract. Scrappy’s document API, chat API, ingestion jobs, and future admin tooling can depend on the auth service instead of duplicating account logic.

The stack

The auth API is built with FastAPI because I wanted an async Python backend that is easy to structure, test, and extend.

PostgreSQL stores persistent user and auth data. SQLAlchemy provides the database access layer, while Alembic handles schema migrations. Redis supports fast session-related operations, including token blacklisting and other short-lived auth checks. Docker Compose makes the local development environment easier to run consistently.

The stack looks roughly like this:

Client App
   |
   | login / verify / logout
   v
Auth API - FastAPI
   |
   | persistent user/account data
   v
PostgreSQL

Auth API
   |
   | token/session checks
   v
Redis

The important part is not just the tooling. It is the boundary.

The auth API owns authentication concerns. Other services can consume identity without owning the full login and account-management flow.

The basic request flow

The core login flow is intentionally simple.

A user signs into a client application. The client sends credentials to the auth API. The auth API validates the credentials, creates a signed JWT, and returns that token to the client. The client then includes that token in future protected requests.

User logs in
   |
Client sends credentials to Auth API
   |
Auth API validates credentials
   |
Auth API returns JWT access token
   |
Client sends token with protected requests
   |
Backend verifies token before allowing access

For protected routes, downstream services can either verify the token directly using the shared signing configuration or call the auth API to validate the token and retrieve the authenticated identity.

That gives the system flexibility. Smaller internal services can delegate verification to the auth API. Higher-throughput services can validate JWTs directly if needed.

Why JWTs make sense here

JWTs are a good fit for this stage of Scrappy because they let authenticated requests remain mostly stateless. Once the user logs in, the client can include the token with future requests, and the backend can verify the token without looking up a full session record every time.

That does not mean Redis becomes unnecessary. Redis still matters for logout, blacklisting, short-lived session checks, and future caching needs.

The combination gives the system a practical balance:

JWTs provide portable signed identity claims. Redis provides fast invalidation and session support. PostgreSQL remains the source of truth for durable user records. Designing for future services

One of the reasons I wanted this as a separate API is that Scrappy is not finished as a single application. It is becoming a small platform.

The document upload flow will need to know which user owns a file. The chunking and embedding pipeline will need to associate processed chunks with the correct account. The chat system will need to retrieve only the documents a user is allowed to access. Admin tools may eventually need role-based permissions.

Those concerns all depend on identity.

By setting up the auth API first, I created a foundation that future services can build on without reworking the security model every time a new feature is added.

What I learned from this part of the build

The biggest lesson from this phase was that auth is not just login.

Login is the visible part. The more important design question is where identity lives in the architecture.

For Scrappy, I wanted account handling to be boring, predictable, and reusable. I wanted one place to manage authentication, one place to issue and validate tokens, and one place for other services to trust user identity.

That makes the rest of the project easier to reason about.

The next pieces of Scrappy can now build on top of this foundation: document storage, ingestion jobs, chunking, embeddings, retrieval, and chat. Each of those features will need user context. The auth API gives them that context in a consistent way.

This post is the first in a longer series on building Scrappy. The auth API is not the flashiest part of the project, but it is the part that lets everything else become a real multi-user system instead of a local prototype.