Immich Feature Implementation Plan

Overview

This document outlines the implementation plan for two highly-requested features in Immich:

Shared Album Photos in Timeline (#1779) - Allow users to see photos from albums shared with them in their main timeline
Search Results Sorted by Date (#8377) - Enable chronological sorting of search results instead of only relevance-based ordering

Feature 1: Shared Album Photos in Timeline

Problem Statement

Currently, when a user shares an album with another user, those photos only appear in the shared album view—never in the recipient's main timeline. This creates friction for family use cases where users want a unified chronological view of all relevant photos.

Proposed Solution

Add a per-album toggle that allows recipients to include a shared album's assets in their personal timeline. The setting is controlled by the recipient, not the sharer.

Epic 1.1: Database Schema & Core Model Changes

Scope: Extend the data model to support timeline inclusion preferences for shared albums.

Stories

Story	Description	Estimate
1.1.1	Add `show_in_timeline` boolean column to `album_users` junction table (default: `false`)	S
1.1.2	Create database migration for the new column	S
1.1.3	Update `AlbumUser` entity in server to include the new field	S
1.1.4	Add index on `(user_id, show_in_timeline)` for efficient timeline queries	S

Files Affected

server/src/entities/album-user.entity.ts
server/src/migrations/ (new migration file)
server/src/repositories/album.repository.ts

Epic 1.2: API Layer Changes

Scope: Expose the timeline preference via REST API endpoints.

Stories

Story	Description	Estimate
1.2.1	Add `showInTimeline` field to `AlbumUserResponseDto`	S
1.2.2	Create `UpdateAlbumUserDto` with `showInTimeline` property	S
1.2.3	Add `PATCH /albums/{albumId}/users/{userId}` endpoint to update user preferences	M
1.2.4	Update OpenAPI spec and regenerate SDK clients (TypeScript, Dart)	S
1.2.5	Add authorization check: only the recipient can modify their own preference	S

Files Affected

server/src/controllers/album.controller.ts
server/src/services/album.service.ts
server/src/dtos/album.dto.ts
open-api/immich-openapi-specs.json

Epic 1.3: Timeline Query Modifications

Scope: Modify the timeline/asset retrieval logic to include shared album assets when enabled.

Stories

Story	Description	Estimate
1.3.1	Extend `getTimeBuckets` query to include shared assets with `show_in_timeline=true`	L
1.3.2	Extend `getTimeBucket` (single bucket) to include shared assets	L
1.3.3	Deduplicate assets that appear both in user's library and shared albums	M
1.3.4	Add visual indicator for shared assets in timeline (metadata flag)	M
1.3.5	Handle permission checks for shared asset operations (view-only vs edit)	M
1.3.6	Performance optimization: ensure query uses proper indexes	M

Key Query Changes

-- Conceptual: Get assets for timeline including shared albums
SELECT a.* FROM assets a
WHERE a.owner_id = :userId
   OR a.id IN (
     SELECT aa.asset_id FROM album_assets aa
     JOIN album_users au ON au.album_id = aa.album_id
     WHERE au.user_id = :userId AND au.show_in_timeline = true
   )
ORDER BY a.local_date_time DESC

Files Affected

server/src/repositories/asset.repository.ts
server/src/services/asset.service.ts
server/src/services/timeline.service.ts (if exists)

Epic 1.4: Web Interface Updates

Scope: Add UI controls for managing timeline inclusion and display shared assets.

Stories

Story	Description	Estimate
1.4.1	Add "Show in Timeline" toggle to shared album settings menu	M
1.4.2	Create album settings modal/panel for shared albums (recipient view)	M
1.4.3	Add visual badge/indicator for shared assets in timeline grid	S
1.4.4	Update timeline store to handle mixed ownership assets	M
1.4.5	Handle click behavior: shared assets open in read-only mode unless user has edit rights	S

Files Affected

web/src/lib/components/album-page/ (album settings)
web/src/lib/components/photos-page/ (timeline view)
web/src/lib/components/asset-viewer/ (viewer modifications)
web/src/lib/stores/ (state management)

Epic 1.5: Mobile App Updates

Scope: Implement timeline inclusion feature in Flutter mobile app.

Stories

Story	Description	Estimate
1.5.1	Regenerate OpenAPI client with new endpoints	S
1.5.2	Add "Show in Timeline" toggle to shared album options	M
1.5.3	Update timeline provider to fetch mixed assets	L
1.5.4	Add visual indicator for shared assets in grid	S
1.5.5	Update local Isar database schema if caching timeline	M
1.5.6	Handle offline sync for shared asset preferences	M

Files Affected

mobile/lib/repositories/ (data layer)
mobile/lib/providers/ (Riverpod providers)
mobile/lib/widgets/asset_grid/ (grid display)
mobile/lib/pages/album/ (album settings)

Epic 1.6: Testing & Documentation

Story	Description	Estimate
1.6.1	Unit tests for album service preference updates	M
1.6.2	E2E tests for timeline with shared assets	L
1.6.3	Mobile widget tests for new UI components	M
1.6.4	Update user documentation with new feature	S
1.6.5	Performance benchmarks with large shared libraries	M

Feature 2: Search Results Sorted by Date

Problem Statement

CLIP-based smart search returns results ordered by relevance score. While useful for finding specific images, users often need chronological ordering—especially when filtering by person, location, or metadata. The current behavior makes it difficult to browse search results temporally.

Technical Challenge

CLIP search works by computing similarity scores against all assets. Every photo gets a score, creating a continuous distribution. Setting a universal cutoff threshold is difficult because optimal thresholds vary per query (as demonstrated with "beach" vs "kite" examples in the discussion).

Proposed Solution

Implement a hybrid approach:

Non-CLIP searches (metadata, tags, people, date filters): Default to chronological order
CLIP searches: Add a "Sort by Date" toggle that applies a relevance threshold, then sorts chronologically
Threshold control: Allow users to specify "top N results" or a relative percentile cutoff

Epic 2.1: Backend Search Enhancements

Scope: Modify search infrastructure to support sorting options and result limiting.

Stories

Story	Description	Estimate
2.1.1	Add `sortBy` enum to `SmartSearchDto`: `RELEVANCE`, `DATE_DESC`, `DATE_ASC`	S
2.1.2	Add `maxResults` optional parameter to limit CLIP results before sorting	S
2.1.3	Modify smart search repository to apply sorting after relevance filtering	L
2.1.4	For non-CLIP searches (metadata only), default to date ordering	M
2.1.5	Add relevance score threshold option (`minScore` parameter)	M
2.1.6	Return relevance score in response for potential UI display	S
2.1.7	Update OpenAPI spec with new search parameters	S

Implementation Notes

// SmartSearchDto additions
interface SmartSearchDto {
  query: string;
  sortBy?: 'relevance' | 'date_desc' | 'date_asc';  // NEW
  maxResults?: number;  // NEW: limit results before sorting
  minScore?: number;    // NEW: relevance threshold (0-1)
  // ... existing filters
}

Query Strategy

-- For date sorting with CLIP:
-- 1. Get top N by relevance from vector search
-- 2. Apply minScore filter if provided  
-- 3. Re-sort by date
WITH ranked AS (
  SELECT *, 1 - (embedding <=> :query_embedding) as score
  FROM smart_search
  WHERE user_id = :userId
  ORDER BY score DESC
  LIMIT :maxResults
)
SELECT * FROM ranked
WHERE score >= :minScore
ORDER BY local_date_time DESC

Files Affected

server/src/dtos/search.dto.ts
server/src/services/search.service.ts
server/src/repositories/search.repository.ts
machine-learning/ (may need changes if relevance score not currently exposed)

Epic 2.2: Differentiate Search vs Filter Behavior

Scope: Implement distinct behavior for "smart search" (CLIP) vs "filter" (metadata-only) queries.

Stories

Story	Description	Estimate
2.2.1	Detect when query contains only filter criteria (no CLIP text)	M
2.2.2	For filter-only queries, bypass CLIP and query directly with date ordering	M
2.2.3	Add `searchMode` indicator in response: `smart` vs `filter`	S
2.2.4	Support combining CLIP search with date range filter	M

User Experience

"Show me photos with person X from 2023" → Filter mode, date-ordered
"beach sunset" → Smart search mode, relevance-ordered (with sort option)
"beach sunset" + person filter → Hybrid: CLIP + filter, user-chosen sort

Epic 2.3: Web Interface Search UI

Scope: Add sorting controls and improve search result presentation.

Stories

Story	Description	Estimate
2.3.1	Add sort dropdown to search results: "Relevance" / "Newest First" / "Oldest First"	M
2.3.2	Add date grouping headers to search results (similar to timeline)	L
2.3.3	Display "Showing top N results" indicator when limit applied	S
2.3.4	Add "Show more results" button to expand beyond initial limit	M
2.3.5	Remember user's preferred sort order in session/settings	S
2.3.6	Add relevance threshold slider (advanced search)	M
2.3.7	Show relevance score badge on hover (optional setting)	S

UI Mockup Concept

┌─────────────────────────────────────────────────────┐
│ 🔍 beach vacation                        [Filters ▾]│
├─────────────────────────────────────────────────────┤
│ Sort by: [Newest First ▾]    Showing 247 results    │
├─────────────────────────────────────────────────────┤
│ ── July 2024 ──────────────────────────────────────│
│ [img] [img] [img] [img] [img]                       │
│ ── March 2023 ─────────────────────────────────────│
│ [img] [img] [img] [img] [img] [img] [img]          │
│ ...                                                 │
└─────────────────────────────────────────────────────┘

Files Affected

web/src/routes/(user)/search/ (search page)
web/src/lib/components/shared-components/search-bar/
web/src/lib/components/photos-page/asset-grid.svelte

Epic 2.4: Mobile App Search Updates

Scope: Implement sorting controls in Flutter app.

Stories

Story	Description	Estimate
2.4.1	Regenerate OpenAPI client	S
2.4.2	Add sort selector to search results screen	M
2.4.3	Implement date grouping in search result grid	L
2.4.4	Add "Load more" functionality with same sort order	M
2.4.5	Persist sort preference in local settings	S

Files Affected

mobile/lib/pages/search/ (search pages)
mobile/lib/providers/search.provider.dart
mobile/lib/widgets/ (reusable components)

Epic 2.5: Advanced Threshold & Cutoff Mechanisms

Scope: Implement intelligent result limiting for CLIP searches.

Stories

Story	Description	Estimate
2.5.1	Implement percentile-based cutoff (e.g., "top 10% of scores")	M
2.5.2	Implement "knee detection" algorithm for automatic threshold	L
2.5.3	Add admin setting for default result limit	S
2.5.4	Expose "Pick cutoff point" UI: user taps image to set threshold	L
2.5.5	Cache threshold decisions per query pattern	M

Knee Detection Algorithm

Find where the score curve has maximum curvature (elbow method):

# Conceptual approach
scores = [0.95, 0.93, 0.91, 0.88, 0.72, 0.71, 0.70, ...]  
# Find index where gradient changes most sharply
knees = find_knee_points(scores)
cutoff_index = knees[0]  # First major drop

Epic 2.6: Testing & Performance

Story	Description	Estimate
2.6.1	Unit tests for sort order application	M
2.6.2	E2E tests for search with different sort modes	L
2.6.3	Performance test: date-sorted CLIP search on 100k+ assets	M
2.6.4	Benchmark query times with/without maxResults limit	M
2.6.5	Mobile integration tests	M
2.6.6	Update API documentation	S

Implementation Priority & Dependencies

Phase 1: Foundation (Weeks 1-3)

Epic 1.1 (Database schema) → Epic 1.2 (API) — No dependencies
Epic 2.1 (Search backend) — No dependencies

Phase 2: Core Features (Weeks 4-7)

Epic 1.3 (Timeline queries) — Depends on 1.1, 1.2
Epic 2.2 (Search vs Filter) — Depends on 2.1
Epic 2.3 (Web search UI) — Depends on 2.1, 2.2

Phase 3: Client Implementation (Weeks 8-11)

Epic 1.4 (Web UI for albums) — Depends on 1.2, 1.3
Epic 1.5 (Mobile albums) — Depends on 1.2, 1.3
Epic 2.4 (Mobile search) — Depends on 2.1, 2.2

Phase 4: Polish & Advanced (Weeks 12-14)

Epic 2.5 (Advanced thresholds) — Depends on 2.1, 2.3
Epic 1.6, 2.6 (Testing) — Depends on respective features

Technical Risks & Mitigations

Risk	Impact	Mitigation
Timeline query performance with many shared albums	High	Add materialized view or denormalized index for timeline assets
CLIP date sorting negates relevance value	Medium	Default to "top N" limit; always show relevance indicator
Mobile sync complexity for shared asset preferences	Medium	Treat preference as server-authoritative; cache locally
Database migration on large instances	Medium	Make migration non-blocking; add column with default
Conflicting deduplication (same asset in library + shared)	Low	Use asset ID for dedup; prefer owned copy's metadata

Success Metrics

Feature 1: Shared Albums in Timeline

Users can enable timeline inclusion for 100% of shared albums
Timeline query time increases < 20% with typical shared album usage
Zero reported data leakage (permissions respected)

Feature 2: Search Sorting

Search results can be sorted in < 500ms for 50k asset libraries
Date-grouped results show correct chronological ordering
User sort preference persists across sessions

Appendix: Component Architecture Reference

┌──────────────────────────────────────────────────────────────┐
│                         CLIENTS                               │
│  ┌─────────┐    ┌─────────┐    ┌─────────┐                   │
│  │   Web   │    │ Mobile  │    │   CLI   │                   │
│  │(Svelte) │    │(Flutter)│    │  (npm)  │                   │
│  └────┬────┘    └────┬────┘    └────┬────┘                   │
│       └──────────────┼──────────────┘                        │
│                      ▼                                        │
│              OpenAPI SDK (auto-generated)                     │
└──────────────────────┬───────────────────────────────────────┘
                       │ REST API
┌──────────────────────▼───────────────────────────────────────┐
│                    SERVER (Nest.js)                           │
│  ┌────────────────────────────────────────────────────────┐  │
│  │ Controllers: album.controller, search.controller, ...  │  │
│  └────────────────────────┬───────────────────────────────┘  │
│                           ▼                                   │
│  ┌────────────────────────────────────────────────────────┐  │
│  │ Services: album.service, search.service, asset.service │  │
│  └────────────────────────┬───────────────────────────────┘  │
│                           ▼                                   │
│  ┌────────────────────────────────────────────────────────┐  │
│  │ Repositories: album.repo, search.repo, asset.repo      │  │
│  └─
───────────────────────┬───────────────────────────────┘  │
└──────────────────────────┬───────────────────────────────────┘
                           │
         ┌─────────────────┼─────────────────┐
         ▼                 ▼                 ▼
    ┌─────────┐      ┌──────────┐      ┌─────────┐
    │PostgreSQL│      │  Redis   │      │   ML    │
    │(+ pgvector)│    │ (jobs)   │      │ (CLIP)  │
    └─────────┘      └──────────┘      └─────────┘

Document Version: 1.0
Based on discussions #1779 and #8377 as of December 2025

Written with StackEdit.

Phlogi/immich.md

Immich Feature Implementation Plan

Overview

Feature 1: Shared Album Photos in Timeline

Problem Statement

Proposed Solution

Epic 1.1: Database Schema & Core Model Changes

Stories

Files Affected

Epic 1.2: API Layer Changes

Stories

Files Affected

Epic 1.3: Timeline Query Modifications

Stories

Key Query Changes

Files Affected

Epic 1.4: Web Interface Updates

Stories

Files Affected

Epic 1.5: Mobile App Updates

Stories

Files Affected

Epic 1.6: Testing & Documentation

Feature 2: Search Results Sorted by Date

Problem Statement

Technical Challenge

Proposed Solution

Epic 2.1: Backend Search Enhancements

Stories

Implementation Notes

Query Strategy

Files Affected

Epic 2.2: Differentiate Search vs Filter Behavior

Stories

User Experience

Epic 2.3: Web Interface Search UI

Stories

UI Mockup Concept

Files Affected

Epic 2.4: Mobile App Search Updates

Stories

Files Affected

Epic 2.5: Advanced Threshold & Cutoff Mechanisms

Stories

Knee Detection Algorithm

Epic 2.6: Testing & Performance

Implementation Priority & Dependencies

Phase 1: Foundation (Weeks 1-3)

Phase 2: Core Features (Weeks 4-7)

Phase 3: Client Implementation (Weeks 8-11)

Phase 4: Polish & Advanced (Weeks 12-14)

Technical Risks & Mitigations

Success Metrics

Feature 1: Shared Albums in Timeline

Feature 2: Search Sorting

Appendix: Component Architecture Reference