Scalable AI Recommendation System

Project Objective: Design and deploy a scalable AI recommendation system combining KNN-based candidate retrieval with rank-fusion re-ranking, targeting 10K+ daily active users and a 35% CTR improvement.

warning

If 10K+ DAU and 35% CTR lift are not real measured results, describe them as targets rather than achieved results on a resume.

---

1. What Problem Are We Solving?

An e-commerce platform may have thousands or millions of products.

When a customer opens the application, we need to answer:

"Which products should we show this customer right now?"

Showing every product is impossible.

Instead, the recommendation system:

Code Block

Customer
   ↓
Understand customer interests
   ↓
Retrieve relevant products
   ↓
Rank the products
   ↓
Show the best recommendations

Customer ↓ Understand customer interests ↓ Retrieve relevant products ↓ Rank the products ↓ Show the best recommendations

The goal is to show products that are:

• relevant to the customer's interests
• similar to products they interacted with
• popular when appropriate
• strongly supported by their behavior
• ranked highly enough to increase the probability of a click

---

2. High-Level Architecture

The architecture has two major recommendation stages:

Retrieval

Find a manageable set of potentially relevant products.

Re-ranking

Take those candidates and determine which ones should appear first.

---

3. Product Representation

A recommendation system needs a numerical representation of products.

Represent every product as a vector:

Code Block

Product → Vector

Product → Vector

For example:

Code Block

iPhone 15
→ [0.90, 0.80, 0.10, ...]

iPhone 15 → [0.90, 0.80, 0.10, ...]

The dimensions can represent learned semantic/product characteristics.

In a real system, these vectors would generally come from an embedding model rather than being manually created.

The important property is:

Similar products should have nearby representations in vector space.

For example:

Code Block

iPhone
Samsung Galaxy
Google Pixel

iPhone Samsung Galaxy Google Pixel

should be closer to one another than to:

Code Block

MacBook

MacBook

or:

Code Block

Headphones

Headphones

---

4. User Representation

The system also needs to represent the customer's interests.

Suppose the customer has interacted with:

Code Block

iPhone
AirPods
iPad
MacBook

iPhone AirPods iPad MacBook

We can construct a user representation from these interactions.

A simple baseline is:

Code Block

User Vector
=
average(product vectors from interaction history)

User Vector = average(product vectors from interaction history)

Conceptually:

The resulting vector represents the customer's observed interests.

In a production system, this can be made more sophisticated using:

• interaction weights
• recency
• purchase vs. click signals
• session behavior
• learned user embeddings
• sequence models

---

5. Why Do We Need Retrieval?

Imagine the catalog contains:

Code Block

10,000,000 products

10,000,000 products

We don't want the ranking system to perform expensive ranking operations over all 10 million products for every request.

Instead:

KNN acts as a candidate retrieval mechanism.

It reduces the search space.

---

6. KNN Candidate Retrieval

KNN means:

K-Nearest Neighbors

Given a user vector, KNN searches for product vectors that are closest to it.

Conceptually:

The closest products become candidates.

For example:

Code Block

User
 ↓
KNN
 ↓
P12
P43
P87
P102
P145
...

User ↓ KNN ↓ P12 P43 P87 P102 P145 ...

These products are not necessarily the final recommendations.

They are candidates.

---

7. Distance and Similarity

KNN needs a way to determine how close two vectors are.

One common choice is cosine similarity/distance.

Cosine similarity measures the orientation of vectors rather than simply their absolute magnitude.

Conceptually:

Code Block

High similarity
      ↓
Vectors point in similar directions


Low similarity
      ↓
Vectors point in different directions

High similarity ↓ Vectors point in similar directions Low similarity ↓ Vectors point in different directions

If KNN returns cosine distance:

Code Block

smaller distance
      ↓
more similar

smaller distance ↓ more similar

A simple transformation is:

Code Block

similarity = 1 - distance

similarity = 1 - distance

for the cosine-distance convention used by the implementation.

---

8. Candidate Retrieval vs. Ranking

This distinction is extremely important.

KNN answers:

"Which products are worth considering?"

Re-ranking answers:

"Which of these candidates should appear first?"

Therefore:

KNN is therefore the retrieval layer, not the entire recommendation system.

---

9. Multiple Recommendation Signals

A single similarity score may not be enough.

A product can be highly similar to the customer but:

• rarely purchased
• temporarily unpopular
• poorly aligned with recent behavior
• less useful than another candidate

Therefore, the system can create multiple rankings.

Each ranking represents a different view of the candidate set.

---

10. Similarity Ranking

The first ranking uses the KNN similarity score.

It answers:

"Which candidates are most similar to this customer's interests?"

Example:

Code Block

1. Product A
2. Product B
3. Product C
4. Product D
5. Product E

1. Product A 2. Product B 3. Product C 4. Product D 5. Product E

This is particularly useful for:

• similar products
• personalized product discovery
• "customers who viewed this may also like..."
• semantic product matching

---

11. Popularity Ranking

A second ranking can use popularity.

Popularity can be derived from signals such as:

• purchases
• clicks
• views
• conversion rate
• recent demand

Conceptually:

This provides a different perspective from pure similarity.

For example:

Code Block

Similarity says:


A
B
C
D


Popularity says:


C
A
D
B

Similarity says: A B C D Popularity says: C A D B

Neither ranking necessarily has to be considered "wrong."

They are answering different questions.

---

12. User-Behavior Ranking

The system can also calculate a user-specific behavioral relevance score.

For example, the customer may have:

• repeatedly viewed smartphones
• purchased Apple accessories
• recently searched for headphones
• added electronics to their cart

The behavioral signal tries to answer:

"Given this customer's behavior, which candidate appears most relevant?"

This creates another ranking.

Code Block

Behavior ranking:


B
D
A
C

Behavior ranking: B D A C

---

13. Why Multiple Rankings?

Suppose three ranking systems produce:

Code Block

Similarity:


A
B
C
D




Popularity:


C
A
D
B




Behavior:


B
D
A
C

Similarity: A B C D Popularity: C A D B Behavior: B D A C

There is no single obvious winner.

Instead, we want to combine the evidence.

This is where rank fusion comes in.

---

14. Rank Fusion

Rank fusion combines multiple ranked lists into one ranking.

Instead of attempting to directly compare completely different scores:

Code Block

similarity score
popularity score
behavior score

similarity score popularity score behavior score

we can work with their rank positions.

One method is:

Reciprocal Rank Fusion (RRF)

---

15. Reciprocal Rank Fusion

RRF assigns a score based on where an item appears in each ranking.

The formula is:

$$RRF(d)= \sum_i \frac{1}{k+rank_i(d)}$$

Where:

• $d$ = product
• $i$ = ranking system
• $rank_i(d)$ = product's position in that ranking
• $k$ = smoothing constant

The scores from all rankings are added together.

---

16. Why Not Just Use 1 / rank?

We could technically use:

$$\frac{1}{rank}$$

But this makes rank position extremely influential.

For example:

Code Block

Rank 1 → 1.000
Rank 2 → 0.500
Rank 5 → 0.200
Rank 10 → 0.100

Rank 1 → 1.000 Rank 2 → 0.500 Rank 5 → 0.200 Rank 10 → 0.100

The difference between rank 1 and rank 10 is huge.

A single #1 position can dominate the combined result.

---

17. What Does k Do?

RRF instead uses:

$$\frac{1}{k+rank}$$

For example, with:

Code Block

k = 60

k = 60

we get:

Code Block

Rank 1  → 1/61 ≈ 0.01639
Rank 4  → 1/64 ≈ 0.01563
Rank 10 → 1/70 ≈ 0.01429

Rank 1 → 1/61 ≈ 0.01639 Rank 4 → 1/64 ≈ 0.01563 Rank 10 → 1/70 ≈ 0.01429

The differences are much smaller.

Therefore, k reduces the influence of the exact rank position.

It makes RRF more interested in:

consistent presence near the top across multiple rankings

rather than:

being #1 in one ranking.

---

18. Consistency vs. One Excellent Ranking

Consider two products.

Product A

Code Block

Similarity → #1
Popularity → #10

Similarity → #1 Popularity → #10

A is extremely strong according to one signal but weak according to another.

Product B

Code Block

Similarity → #4
Popularity → #4

Similarity → #4 Popularity → #4

B is consistently strong.

Without k

Using:

$$1/rank$$

Product A:

$$1/1 + 1/10 = 1.10$$

Product B:

$$1/4 + 1/4 = 0.50$$

Result:

Code Block

A → recommended
B → not recommended

A → recommended B → not recommended

The #1 ranking dominates.

With k = 60

Product A:

$$1/61 + 1/70 \approx 0.03068$$

Product B:

$$1/64 + 1/64 \approx 0.03125$$

Result:

Code Block

B → recommended
A → not recommended

B → recommended A → not recommended

Now the consistently strong product wins.

---

19. The Core Idea Behind k

k does not change the original rankings.

It changes how strongly rank positions influence the combined ranking.

Think of it as a smoothing parameter:

The commonly used RRF formulation uses k = 60, but k is a configurable hyperparameter rather than a random number.

---

20. Final Ranking

After calculating RRF scores:

Code Block

Product A → 0.0481
Product B → 0.0478
Product C → 0.0469
Product D → 0.0452

Product A → 0.0481 Product B → 0.0478 Product C → 0.0469 Product D → 0.0452

we sort by the fused score.

---

21. Top-N Recommendation

The final system doesn't need to return every candidate.

It can return:

Code Block

Top 3

Top 3

or:

Code Block

Top 5

Top 5

or:

Code Block

Top 10

Top 10

depending on the product experience.

For example:

Code Block

1. Product B
2. Product A
3. Product D

1. Product B 2. Product A 3. Product D

These are what the customer sees.

---

22. End-to-End Pipeline

---

23. Why This Architecture Is Scalable

The main idea is to separate retrieval from ranking.

Instead of:

Code Block

10 million products
       ↓
expensive ranking
       ↓
top 10

10 million products ↓ expensive ranking ↓ top 10

we do:

Code Block

10 million products
       ↓
fast vector retrieval
       ↓
100 candidates
       ↓
more sophisticated ranking
       ↓
top 10

10 million products ↓ fast vector retrieval ↓ 100 candidates ↓ more sophisticated ranking ↓ top 10

This allows the expensive ranking logic to operate on a much smaller set.

As the catalog grows, this separation becomes increasingly important.

---

24. Serving Architecture

A production deployment can expose the recommender through an API.

Example conceptual request:

Code Block

GET /recommendations?user_id=U123

GET /recommendations?user_id=U123

Response:

Code Block

{
  "user_id": "U123",
  "recommendations": [
    "P42",
    "P87",
    "P15"
  ]
}

{ "user_id": "U123", "recommendations": [ "P42", "P87", "P15" ] }

---

25. Production Data Flow (PostgreSQL & pgvector Pipeline)

Our system leverages PostgreSQL and the pgvector extension to completely unify storage and similarity search. This eliminates the need to keep large .pkl indexes in memory and ensures our API remains extremely lightweight.

Architecture Diagram

Components

1. FastAPI Event Logging (api/routes.py): Exposes a POST /events (or similar logging mechanisms) to capture user interactions (clicks, views) into an events.db database.

2. Embeddings Updater (core/trainer.py): The update_user_embeddings() method fetches events from the events database, recalculates dynamic user vectors by averaging product embeddings (fetched from PostgreSQL), and upserts them directly into the PostgreSQL user_embeddings table.

3. Database Layer (core/database.py): Handles PostgreSQL connections using psycopg2. It utilizes pgvector's <=> (cosine distance) operator to perform the KNN search natively inside the database, totally eliminating the need for scikit-learn or .pkl files.

4. Pipeline Orchestrator (main.py): The FastAPI app allows triggering the core/trainer.py pipeline as a background task to keep user vectors fresh based on real-time events.

---

26. Measuring Recommendation Quality

The system should not be evaluated only by whether it produces recommendations.

We need measurable business outcomes.

CTR

$$CTR = \frac{clicks}{impressions}$$

It answers:

"How often do users click recommended products?"

Conversion Rate

$$CVR = \frac{conversions}{clicks}$$

It answers:

"How often does a click result in a conversion?"

Revenue per User

Measures the business impact of recommendations.

---

27. A/B Testing

To claim a CTR improvement, we need an experiment.

A claimed:

35% CTR lift

should mean a measured relative improvement such as:

$$CTR_{new} = 1.35 \times CTR_{baseline}$$

For example:

Code Block

Baseline CTR = 10%


New CTR = 13.5%


Relative lift = 35%

Baseline CTR = 10% New CTR = 13.5% Relative lift = 35%

This is different from saying the CTR itself is 35%.

---

28. Scale Target

The system is designed around a target of:

10K+ daily active users

This means the serving architecture must be able to handle recommendation requests from a significant number of daily users while maintaining acceptable latency.

The important architectural principle is:

Code Block

Offline computation
       +
Efficient online retrieval
       +
Lightweight online ranking

Offline computation + Efficient online retrieval + Lightweight online ranking

rather than performing expensive model computation from scratch for every request.

---

29. What Each Component Does

Component	Responsibility
Product embeddings	Represent products numerically
User representation	Represent customer interests
KNN	Retrieve potentially relevant candidates
Similarity ranking	Rank by vector relevance
Popularity ranking	Rank by popularity
Behavior ranking	Rank by user-specific behavior
RRF	Fuse multiple rankings
Top-N	Select products shown to customer
API	Serve recommendations
A/B testing	Measure business impact

---

30. The Most Important Mental Model

Remember these three questions:

1. Retrieval

"What products should I consider?"

KNN answers this.

2. Re-ranking

"Which of those candidates should appear first?"

Multiple ranking signals answer this.

3. Rank fusion

"How do I combine those different opinions?"

RRF answers this.

So the complete recommendation strategy is:

Vector-based KNN candidate retrieval followed by multi-signal rank-fusion re-ranking.

---

31. Resume-Level Description

If the 35% CTR lift is actually measured, it can be stated directly:

Designed and deployed a scalable AI recommendation system combining KNN-based candidate retrieval with RRF re-ranking, serving 10K+ daily active users and achieving a 35% CTR lift.

If it is only a target and not measured, use:

Designed and deployed a scalable AI recommendation system combining KNN-based candidate retrieval with RRF re-ranking, targeting 10K+ daily active users and improved CTR through multi-signal personalization.

---

32. What You Should Be Able to Explain in an Interview

For a one-day preparation window, focus on these seven things:

1. Why represent products as vectors?
2. How do we turn user history into a user vector?
3. Why use KNN?
4. Why is KNN retrieval rather than the final ranking?
5. Why do we have similarity, popularity, and behavior rankings?
6. Why use RRF instead of simply adding unrelated scores?
7. What does k do, and why can consistency beat being #1 in only one ranking?

If you can explain those clearly, you understand the core of the project rather than merely knowing the code.