# Redis Cache Migration Analysis

**Based on Database Schema Changes for MongoDB Prisma Models**

---

## Overview

With 58,000 videos in VideoMedia and the new multi-category support (categoryIds array), this document outlines all Redis cache key changes, memory implications, and new keys for optimal performance.

---

## Database Changes Summary

| Entity         | Changes                                             | Impact                                                            |
| -------------- | --------------------------------------------------- | ----------------------------------------------------------------- |
| **Channel**    | Add categories & tags as JSON; add tagNames array   | New denormalized data in cache                                    |
| **Category**   | Remove channelId; change tags to JSON; add tagNames | No direct dependency on Channel; tag relationships stored as JSON |
| **Tag**        | Remove channelId; no Channel relationship           | Simpler structure; only tied to Category                          |
| **VideoMedia** | `categoryId` → `categoryIds` (string[])             | Videos can belong to multiple categories                          |

---

## PART 1: REDIS KEYS REQUIRING CHANGES

### 1.1 Channel Cache Keys

#### CHANGE: Remove implicit category/tag relationships

**Current Key Structure:**

```
app:channel:all → ChannelCachePayload[]
```

**Current ChannelCachePayload:**

```typescript
{
  id: string;
  name: string;
  landingUrl: string;
  videoCdn?: string;
  coverCdn?: string;
  clientName?: string;
  clientNotice?: string;
  remark?: string;
}
```

**Updated ChannelCachePayload:**

```typescript
{
  id: string;
  name: string;
  landingUrl: string;
  videoCdn?: string;
  coverCdn?: string;
  clientName?: string;
  clientNotice?: string;
  remark?: string;
  categories?: Array<{ id: string; name: string }>;  // NEW
  tags?: Array<{ id: string; name: string }>;        // NEW
  tagNames?: string[];                               // NEW: tag names only
}
```

**Memory Impact:** +20-40 bytes per channel (minimal, channels < 100)

---

### 1.2 Category Cache Keys

#### CHANGE: Remove channelId; change tags to JSON; add tagNames

**Current Key Structure:**

```
app:category:all → CategoryCachePayload[]
app:category:by-id:{categoryId} → CategoryCachePayload (single)
app:category:with-tags:{categoryId} → CategoryCachePayload with tags
app:tag:list:{categoryId} → Tag[] (JSON array in Redis list)
```

**Current CategoryCachePayload:**

```typescript
{
  id: string;
  name: string;
  subtitle?: string | null;
  channelId: string;           // REMOVE THIS
  seq: number;
  tags: string[];              // Array of tag IDs or names
}
```

**Updated CategoryCachePayload:**

```typescript
{
  id: string;
  name: string;
  subtitle?: string | null;
  seq: number;
  tags?: Array<{ id: string; name: string }>;  // Changed from string[] to JSON objects
  tagNames?: string[];                         // NEW: array of tag names only
  status?: number;                             // OPTIONAL: add status for filtering
}
```

**Affected Keys:**

- `app:category:all`
- `app:category:by-id:{categoryId}`
- `app:category:with-tags:{categoryId}`

**Memory Impact per Category:**

- Old: ~100 bytes (id + name + subtitle + channelId + seq + tag IDs)
- New: ~150-200 bytes (includes tag objects + tagNames)
- **Increase: ~50-100 bytes per category** (assuming ~1000-5000 categories)
- **Total increase: ~50-500 KB**

---

### 1.3 Tag Cache Keys

#### CHANGE: Remove channelId; update TagCachePayload

**Current Key Structure:**

```
app:tag:all → TagCachePayload[]
app:tag:list:{categoryId} → Tag[] (Redis list of JSON objects)
```

**Current TagCachePayload:**

```typescript
{
  id: string;
  name: string;
  channelId: string; // REMOVE THIS
  categoryId: string;
  seq: number;
}
```

**Updated TagCachePayload:**

```typescript
{
  id: string;
  name: string;
  categoryId: string;
  seq: number;
  status?: number;       // OPTIONAL: add status
}
```

**Affected Keys:**

- `app:tag:all`
- `app:tag:list:{categoryId}` (no change to key name, but payload updated)

**Memory Impact per Tag:**

- Old: ~80 bytes (id + name + channelId + categoryId + seq)
- New: ~60 bytes (removed channelId)
- **Savings: ~20 bytes per tag** (assuming ~10,000-50,000 tags)
- **Total savings: ~200-1000 KB**

---

### 1.4 VideoMedia Cache Keys - CRITICAL CHANGE

#### MAJOR CHANGE: Single categoryId → Multiple categoryIds

**Current VideoMedia fields in cache:**

```typescript
{
  id: string;
  title: string;
  // ... 50+ fields from provider ...
  categoryId: string;    // Single category reference (CHANGE THIS)
  tagIds: string[];      // Tag array
  tags: string[];        // Tag names array
  tagsFlat: string;      // Concatenated tag names for search
  listStatus: number;
}
```

**Updated VideoMedia fields in cache:**

```typescript
{
  id: string;
  title: string;
  // ... 50+ fields from provider ...
  categoryIds: string[]; // Multiple category references (NEW)
  tagIds: string[];      // Tag array (unchanged)
  tags: string[];        // Tag names array (unchanged)
  tagsFlat: string;      // Concatenated tag names for search (unchanged)
  listStatus: number;
}
```

**Affected Cache Keys:**

- `app:video:detail:{videoId}` - Each video detail JSON

**Current Size per Video:**

- ~500-800 bytes (approx)

**New Size per Video:**

- ~520-820 bytes (add ~20 bytes for extra category ID)

**Total Memory Impact (58,000 videos):**

- Old: ~58,000 × 650 bytes = **~37.7 MB**
- New: ~58,000 × 670 bytes = **~38.9 MB**
- **Increase: ~1.2 MB**

---

### 1.5 Video List Cache Keys (No Changes Needed)

**Keys remain the same:**

```
app:video:category:list:{categoryId}      → List of video IDs (LRANGE, LIST type)
app:video:tag:list:{categoryId}:{tagId}   → List of video IDs (LRANGE, LIST type)
app:video:list:home:{channelId}:{section} → List of video IDs (ZSET or LIST)
```

**However:** New application logic needed to handle:

- A video appearing in multiple category lists
- Efficient querying of videos across multiple categoryIds

---

## PART 2: NEW REDIS KEYS TO ADD

### 2.1 Video-by-Category Index (for multi-category support)

**Purpose:** Enable efficient lookup of all videos in a set of categories

**Key Pattern:**

```
app:video:category:index:{categoryId}:videos
```

**Type:** ZSET (sorted set with score = addedTime for sorting)

**Value:** Video IDs with score as timestamp

**Use Case:**

```typescript
// Get all videos in multiple categories with pagination
const categoryIds = ['cat1', 'cat2', 'cat3'];
const keys = categoryIds.map((id) => `app:video:category:index:${id}:videos`);

// Use ZUNIONSTORE to combine multiple category indexes
await redis.zunionstore('temp:combined:videos', categoryIds.length, ...keys);

// Paginate: ZREVRANGE 'temp:combined:videos' 0 19
```

**Memory Per Key:** ~100 bytes + 30 bytes per video

**Total Memory (assuming avg 200 videos per category, ~2000 categories):**

- Keys: 2000 × 100 = 200 KB
- Values: 2000 × 200 × 30 = 12 MB
- **Total: ~12.2 MB**

---

### 2.2 Category-to-Video Count Cache (for UI statistics)

**Purpose:** Quick lookup of video count per category without scanning

**Key Pattern:**

```
app:category:video:count:{categoryId}
```

**Type:** STRING (integer count)

**Value:** Integer count

**Use Case:**

```typescript
// Get category with video count for UI display
const count = await redis.get(`app:category:video:count:${categoryId}`);
```

**Memory Per Key:** ~40 bytes (one per category)

**Total Memory (~2000 categories):**

- **~80 KB**

---

### 2.3 Tag Search Index (for efficient tag filtering with names)

**Purpose:** Full-text tag search by partial name matching

**Key Pattern:**

```
app:tag:search:{prefix}
```

**Type:** SET (set of matching tag IDs)

**Value:** Tag IDs matching the prefix

**Use Case:**

```typescript
// Find all tags starting with "act"
const tagIds = await redis.smembers('app:tag:search:act');
```

**Note:** Update on tag name changes or new tags created

**Memory Per Prefix (assuming 2-3 char prefixes, ~500-1000 prefixes):**

- **~500 KB - 1 MB**

---

### 2.4 Video Multi-Category Lookup Index

**Purpose:** Fast lookup of categories for a specific video

**Key Pattern:**

```
app:video:categories:{videoId}
```

**Type:** SET (set of category IDs)

**Value:** Category IDs

**Use Case:**

```typescript
// Get all categories a video belongs to
const categoryIds = await redis.smembers(`app:video:categories:${videoId}`);
```

**Memory Per Key:** ~40 bytes + 30 bytes per category

**Assuming avg 2-3 categories per video:**

- Per key: ~40 + (3 × 30) = ~130 bytes
- Total (58,000 videos): ~7.5 MB

---

### 2.5 Category Tagnames Flat Index (for quick name searches)

**Purpose:** Pre-computed concatenation of all tag names in a category for search

**Key Pattern:**

```
app:category:tagnames:flat:{categoryId}
```

**Type:** STRING

**Value:** Space-separated tag names, lowercase

**Use Case:**

```typescript
// Check if category has tag by name (without JSON parsing)
const tagnames = await redis.get(`app:category:tagnames:flat:${categoryId}`);
const hasTag = tagnames?.includes('action'); // O(1) string search
```

**Memory Per Key:** ~100-300 bytes per category

**Total (~2000 categories):** ~200-600 KB

---

### 2.6 Channel Denormalization Cache (future - optional)

**Key Pattern:**

```
app:channel:categories:{channelId}
app:channel:tags:{channelId}
```

**Type:** SET or JSON

**Note:** If Channel.categories and Channel.tags JSON fields are meant to be denormalized, these keys cache pre-computed sets

**Memory:** ~1-2 MB (if implemented)

---

## PART 3: CACHE INVALIDATION STRATEGY

### 3.1 On Category Update

When a category is updated:

```typescript
// Invalidate
app:category:all
app:category:by-id:{categoryId}
app:category:with-tags:{categoryId}
app:category:video:count:{categoryId}
app:category:tagnames:flat:{categoryId}
```

---

### 3.2 On Tag Update

When a tag is updated:

```typescript
// Invalidate
app: tag: all;
app: tag: list: {
  categoryId;
}
app: tag: search: {
  prefix;
} // All prefixes containing the tag
app: category: tagnames: flat: {
  categoryId;
}
```

---

### 3.3 On Video Update (CategoryIds Change)

When a video's categoryIds array changes:

```typescript
// Invalidate
app:video:detail:{videoId}
app:video:category:index:{categoryId}:videos  // For each old & new category
app:video:categories:{videoId}
app:category:video:count:{categoryId}  // For each affected category
```

---

### 3.4 On Channel Update

When a channel is updated:

```typescript
// Invalidate
app:channel:all
app:channel:by-id:{channelId}
app:channel:with-categories:{channelId}
app:channel:categories:{channelId}
app:channel:tags:{channelId}
```

---

## PART 4: MEMORY ESTIMATION

### Current Memory Usage (Before Changes)

| Component     | Count  | Size/Unit | Total         |
| ------------- | ------ | --------- | ------------- |
| Channels      | ~50    | 200 B     | ~10 KB        |
| Categories    | ~2000  | 150 B     | ~300 KB       |
| Tags          | ~10000 | 80 B      | ~800 KB       |
| Video Details | 58000  | 650 B     | ~37.7 MB      |
| Video Lists   | ~2000  | varies    | ~5-10 MB      |
| **Subtotal**  |        |           | **~44-49 MB** |

### New Memory Usage (After Changes)

| Component                  | Count     | Size/Unit | Total         | Change        |
| -------------------------- | --------- | --------- | ------------- | ------------- |
| Channels                   | ~50       | 250 B     | ~12.5 KB      | +2.5 KB       |
| Categories                 | ~2000     | 200 B     | ~400 KB       | +100 KB       |
| Tags                       | ~10000    | 60 B      | ~600 KB       | -200 KB       |
| Video Details              | 58000     | 670 B     | ~38.9 MB      | +1.2 MB       |
| Video Lists                | ~2000     | varies    | ~5-10 MB      | 0             |
| **Category-Video Index**   | 2000      | varies    | ~12.2 MB      | +12.2 MB      |
| **Video Categories Index** | 58000     | 130 B     | ~7.5 MB       | +7.5 MB       |
| **Category Count Cache**   | 2000      | 40 B      | ~80 KB        | +80 KB        |
| **Tag Search Index**       | ~500-1000 | varies    | ~0.5-1 MB     | +0.5-1 MB     |
| **Category Tagnames Flat** | 2000      | 200 B     | ~400 KB       | +400 KB       |
| **Subtotal**               |           |           | **~77-82 MB** | **+28-33 MB** |

---

## PART 5: RECOMMENDED NEW REDIS KEYS CHECKLIST

### For Multi-Category Support ✅

- [ ] `app:video:category:index:{categoryId}:videos` - ZSET of video IDs by category
- [ ] `app:video:categories:{videoId}` - SET of category IDs per video
- [ ] `app:category:video:count:{categoryId}` - STRING count of videos

### For Search Optimization ✅

- [ ] `app:tag:search:{prefix}` - SET of matching tag IDs
- [ ] `app:category:tagnames:flat:{categoryId}` - STRING of space-separated tag names

### For Quick Metadata Lookups ✅

- [ ] `app:channel:with-categories:{channelId}` - JSON with denormalized categories
- [ ] `app:channel:with-tags:{channelId}` - JSON with denormalized tags

---

## PART 6: TTL (Time-To-Live) RECOMMENDATIONS

### Short-lived (refresh on every update)

- `app:category:video:count:{categoryId}` - TTL: 3600 (1 hour)
- `app:video:category:index:{categoryId}:videos` - TTL: 7200 (2 hours)
- `app:tag:search:{prefix}` - TTL: 7200 (2 hours)

### Medium-lived (cache for 24 hours)

- `app:video:detail:{videoId}` - TTL: 86400 (24 hours)
- `app:video:categories:{videoId}` - TTL: 86400 (24 hours)
- `app:category:tagnames:flat:{categoryId}` - TTL: 86400 (24 hours)

### Long-lived (cache for 7 days)

- `app:channel:all` - TTL: 604800 (7 days)
- `app:category:all` - TTL: 604800 (7 days)
- `app:tag:all` - TTL: 604800 (7 days)

---

## PART 7: IMPLEMENTATION PRIORITY

### Phase 1 (Critical - Do First)

1. Update `VideoMedia` cache to use `categoryIds[]` instead of `categoryId`
2. Update video detail key serialization
3. Add database migration for categoryIds field

### Phase 2 (High - Build Index Support)

1. Add `app:video:category:index:{categoryId}:videos` index builder
2. Add `app:video:categories:{videoId}` lookup cache
3. Add `app:category:video:count:{categoryId}` counter

### Phase 3 (Medium - Optimization)

1. Add `app:category:tagnames:flat:{categoryId}` for search
2. Add `app:tag:search:{prefix}` for tag autocomplete
3. Update Channel/Category cache builders

### Phase 4 (Optional - Polish)

1. Add denormalization caches for quick channel lookups
2. Implement cache warming strategies
3. Add monitoring/alerting for cache hit rates

---

## PART 8: SAMPLE CACHE BUILDER UPDATES

### Updated TagCacheBuilder

```typescript
export interface TagCachePayload {
  id: string;
  name: string;
  categoryId: string;  // Changed: removed channelId
  seq: number;
  status?: number;     // New: optional
}

async buildAll(): Promise<void> {
  const tags = await this.mongoPrisma.tag.findMany({
    where: { status: 1 },
    orderBy: [{ seq: 'asc' }, { name: 'asc' }],
  });

  const payload: TagCachePayload[] = tags.map((tag) => ({
    id: tag.id,
    name: tag.name,
    categoryId: tag.categoryId,  // Only categoryId now
    seq: tag.seq,
    status: tag.status,
  }));

  await this.redis.setJson(CacheKeys.appTagAll, payload);
  this.logger.log(`Built ${payload.length} tags`);
}
```

### Updated CategoryCacheBuilder

```typescript
export interface CategoryCachePayload {
  id: string;
  name: string;
  subtitle?: string | null;
  seq: number;
  tags?: Array<{ id: string; name: string }>;  // Changed: JSON objects
  tagNames?: string[];                         // New: tag names only
  status?: number;                             // New: optional status
}

async buildAll(): Promise<void> {
  const categories = await this.mongoPrisma.category.findMany({
    where: { status: 1 },
    orderBy: [{ seq: 'asc' }, { name: 'asc' }],
  });

  const categoryIds = categories.map((c) => c.id);
  const tags = categoryIds.length
    ? await this.mongoPrisma.tag.findMany({
        where: { status: 1, categoryId: { in: categoryIds } },
        orderBy: [{ seq: 'asc' }, { name: 'asc' }],
      })
    : [];

  // Build tag lookup
  const tagsByCategory = new Map<string, Array<{ id: string; name: string }>>();
  const tagNamesByCategory = new Map<string, string[]>();

  for (const tag of tags) {
    // JSON objects
    const list = tagsByCategory.get(tag.categoryId) || [];
    list.push({ id: tag.id, name: tag.name });
    tagsByCategory.set(tag.categoryId, list);

    // Tag names only
    const nameList = tagNamesByCategory.get(tag.categoryId) || [];
    nameList.push(tag.name);
    tagNamesByCategory.set(tag.categoryId, nameList);
  }

  const payloads: CategoryCachePayload[] = categories.map((category) => ({
    id: category.id,
    name: category.name,
    subtitle: category.subtitle ?? null,
    seq: category.seq,
    tags: tagsByCategory.get(category.id),        // JSON objects
    tagNames: tagNamesByCategory.get(category.id), // Names only
    status: category.status,
  }));

  await this.redis.setJson(CacheKeys.appCategoryAll, payloads);
  this.logger.log(`Built ${payloads.length} categories with tags`);
}
```

### New: VideoCategoryIndexBuilder

```typescript
export class VideoCategoryIndexBuilder {
  async buildAll(): Promise<void> {
    const videos = await this.mongoPrisma.videoMedia.findMany({
      select: { id: true, categoryIds: true, addedTime: true },
      where: { listStatus: 1 },
    });

    // Group videos by each category
    const categoryIndex = new Map<
      string,
      Array<{ id: string; score: number }>
    >();

    for (const video of videos) {
      const score = video.addedTime?.getTime() || 0;
      for (const categoryId of video.categoryIds) {
        const list = categoryIndex.get(categoryId) || [];
        list.push({ id: video.id, score });
        categoryIndex.set(categoryId, list);
      }
    }

    // Store in Redis as ZSET
    for (const [categoryId, videoList] of categoryIndex.entries()) {
      const key = `app:video:category:index:${categoryId}:videos`;
      const members = videoList.map((v) => ({ member: v.id, score: v.score }));
      await this.redis.zadd(key, ...members.flat());
      await this.redis.expire(key, 7200); // 2 hours TTL
    }

    this.logger.log(
      `Built category video index for ${categoryIndex.size} categories`,
    );
  }
}
```

---

## Summary Table: All Changes

| Layer                   | Old                    | New                                   | Impact                       |
| ----------------------- | ---------------------- | ------------------------------------- | ---------------------------- |
| **Database**            | Category has channelId | Category independent                  | DB schema cleaner            |
| **Database**            | VideoMedia.categoryId  | VideoMedia.categoryIds                | Support multi-category       |
| **Database**            | Tag has channelId      | Tag no Channel link                   | Simplified relationships     |
| **Cache: Channel**      | Basic fields only      | + categories, tags as JSON + tagNames | +20-40B per channel          |
| **Cache: Category**     | + channelId            | - channelId; tags as JSON; + tagNames | -20B but +50-100B for JSON   |
| **Cache: Tag**          | + channelId            | - channelId                           | -20B per tag                 |
| **Cache: Video Detail** | categoryId             | categoryIds                           | +20B per video; +1.2MB total |
| **Cache: Indexes**      | N/A                    | + category-video index (ZSET)         | +12.2 MB                     |
| **Cache: Indexes**      | N/A                    | + video-categories lookup             | +7.5 MB                      |
| **Cache: Indexes**      | N/A                    | + category count cache                | +80 KB                       |
| **Total Memory**        | ~44-49 MB              | ~77-82 MB                             | +33 MB (66% increase)        |

---

## Estimated Timeline & Effort

| Phase       | Tasks                                         | Effort | Time     |
| ----------- | --------------------------------------------- | ------ | -------- |
| **Phase 1** | Update Prisma, migration, video detail cache  | High   | 2-3 days |
| **Phase 2** | Build index builders, implement lookup caches | Medium | 2-3 days |
| **Phase 3** | Add search indexes, flatname caches           | Medium | 1-2 days |
| **Phase 4** | Denormalization, monitoring, testing          | Low    | 1 day    |
| **Total**   |                                               |        | 6-9 days |

---

## Notes & Recommendations

1. **Redis Memory**: Ensure Redis instance can handle 80+ MB (consider Redis Cluster if needed)
2. **Index Freshness**: Implement background jobs to refresh category-video indexes every 2 hours
3. **Monitoring**: Add cache hit rate monitoring and alert on low hit rates
4. **Testing**: Load test with 58K+ videos to validate performance
5. **Backward Compatibility**: Ensure old cache keys are invalidated during migration
6. **Documentation**: Update CACHE_SEMANTICS.md with new key patterns and semantics