joshpatra/allstarr
1
0
Fork 0
mirror of https://github.com/SoPat712/allstarr.git synced 2026-02-09 23:55:10 -05:00
Code Issues Actions 2 Packages Projects Releases Wiki Activity

refactor: remove sync window logic from Spotify Import

Browse Source 
- Simplified SpotifyMissingTracksFetcher to remove complex sync window timing
- Now fetches on startup if cache missing, then checks every 5 minutes for stale cache (>24h)
- Removed SYNC_START_HOUR, SYNC_START_MINUTE, SYNC_WINDOW_HOURS from config
- Updated README and .env.example to reflect simpler configuration
- Sync window was only relevant for legacy Jellyfin plugin scraping method
- When using sp_dc cookie method (recommended), this service is dormant anyway
- Deleted MIGRATION.md (local-only file, not for repo)
This commit is contained in:
Josh Patra
2026-02-08 01:21:45 -05:00
parent baab1e88a5
commit 8dbf37f6a3
8 changed files with 50 additions and 375 deletions
Download Patch File Download Diff File Expand all files Collapse all files

12
.env.example
View File

@@ -116,18 +116,6 @@ CACHE_DURATION_HOURS=1
# Enable Spotify playlist injection (optional, default: false)
SPOTIFY_IMPORT_ENABLED=false
# Sync schedule: When does the Spotify Import plugin run?
# Used for the sync window check to avoid fetching too frequently
# Set these to match your plugin's sync schedule in Jellyfin
# Example: If plugin runs daily at 4:15 PM, set HOUR=16 and MINUTE=15
SPOTIFY_IMPORT_SYNC_START_HOUR=16
SPOTIFY_IMPORT_SYNC_START_MINUTE=15
# Sync window: How long to search for missing tracks files (in hours)
# The fetcher will check within this window after the sync start time
# Example: If plugin runs at 4:15 PM and window is 2 hours, checks from 4:15 PM to 6:15 PM
SPOTIFY_IMPORT_SYNC_WINDOW_HOURS=2
# Matching interval: How often to run track matching (in hours)
# Spotify playlists like Discover Weekly update once per week, Release Radar updates weekly
# Set to 0 to only run once on startup (manual trigger via admin UI still works)

3
.gitignore vendored
View File

@@ -104,3 +104,6 @@ originals/
# Sample missing playlists for Spotify integration testing
sampleMissingPlaylists/
# Migration guide (local only)
MIGRATION.md

157
MIGRATION.md
View File

@@ -1,157 +0,0 @@
# Migration Guide: Reorganizing Download Structure
This guide is for upgrading from the old download structure to the new unified structure.
## Old Structure
```
./downloads/ # Permanent downloads
./kept/ # Favorited tracks
./cache/ # Cached tracks
```
## New Structure
```
./downloads/
├── permanent/ # Permanent downloads
├── kept/ # Favorited tracks
└── cache/ # Cached tracks
```
## Migration Steps
### 1. Stop the Container
```bash
docker-compose down
```
### 2. Backup Your Data (Recommended)
```bash
# Create a backup
tar -czf allstarr-backup-$(date +%Y%m%d).tar.gz downloads/ kept/ cache/ 2>/dev/null
```
### 3. Create New Directory Structure
```bash
mkdir -p downloads/permanent downloads/kept downloads/cache
```
### 4. Move Existing Files
**Move permanent downloads:**
```bash
# If you have files directly in downloads/
if [ -d "downloads" ] && [ ! -d "downloads/permanent" ]; then
# Move all files/folders except the new subdirectories
find downloads/ -maxdepth 1 -mindepth 1 ! -name 'permanent' ! -name 'kept' ! -name 'cache' ! -name 'playlists' -exec mv {} downloads/permanent/ \;
fi
# Move playlists folder if it exists
if [ -d "downloads/playlists" ]; then
mv downloads/playlists downloads/permanent/
fi
```
**Move kept files:**
```bash
if [ -d "kept" ]; then
mv kept/* downloads/kept/ 2>/dev/null || true
rmdir kept
fi
```
**Move cache files:**
```bash
if [ -d "cache" ]; then
mv cache/* downloads/cache/ 2>/dev/null || true
rmdir cache
fi
```
### 5. Update .env File
```bash
# Remove old variables
sed -i.bak '/^KEPT_PATH=/d' .env
sed -i.bak '/^CACHE_PATH=/d' .env
# Ensure DOWNLOAD_PATH is set correctly
if ! grep -q "^DOWNLOAD_PATH=" .env; then
echo "DOWNLOAD_PATH=./downloads" >> .env
else
sed -i.bak 's|^DOWNLOAD_PATH=.*|DOWNLOAD_PATH=./downloads|' .env
fi
```
### 6. Update Media Server Library Paths
**For Jellyfin:**
1. Go to Dashboard → Libraries
2. Edit your Music library
3. Update the folder path from `downloads` to `downloads/permanent`
4. Scan library
**For Navidrome/Subsonic:**
1. Update your music folder configuration
2. Change from `downloads` to `downloads/permanent`
3. Restart and rescan
### 7. Verify Migration
```bash
# Check the new structure
ls -la downloads/
ls -la downloads/permanent/
ls -la downloads/kept/
ls -la downloads/cache/
# Count files in each directory
echo "Permanent: $(find downloads/permanent -type f | wc -l) files"
echo "Kept: $(find downloads/kept -type f | wc -l) files"
echo "Cache: $(find downloads/cache -type f | wc -l) files"
```
### 8. Start the Container
```bash
docker-compose up -d
```
### 9. Check Logs
```bash
docker-compose logs -f allstarr
```
## Rollback (If Needed)
If something goes wrong:
```bash
# Stop container
docker-compose down
# Restore from backup
tar -xzf allstarr-backup-YYYYMMDD.tar.gz
# Restore old .env
mv .env.bak .env
# Start container
docker-compose up -d
```
## Verification Checklist
- [ ] All files moved to new structure
- [ ] Old directories removed or empty
- [ ] .env file updated
- [ ] Media server library paths updated
- [ ] Container starts without errors
- [ ] Can play existing tracks
- [ ] New downloads go to correct folders
- [ ] Favoriting external tracks works
- [ ] Cache cleanup works
## Notes
- The migration preserves all your existing files
- Playlists (.m3u files) are moved to `downloads/permanent/playlists/`
- Relative paths in M3U files should still work
- If you have a lot of files, the migration may take a few minutes
- The backup is optional but highly recommended

8
README.md
View File

@@ -360,9 +360,6 @@ Allstarr automatically fills your Spotify playlists (like Release Radar and Disc
| Setting | Description |
|---------|-------------|
| `SpotifyImport:Enabled` | Enable Spotify playlist injection (default: `false`) |
| `SpotifyImport:SyncStartHour` | Hour when plugin runs (24-hour format, 0-23) - used for sync window check |
| `SpotifyImport:SyncStartMinute` | Minute when plugin runs (0-59) - used for sync window check |
| `SpotifyImport:SyncWindowHours` | Hours to check for missing tracks files after sync time (default: 2) |
| `SpotifyImport:MatchingIntervalHours` | How often to run track matching in hours (default: 24, set to 0 for startup only) |
| `SpotifyImport:Playlists` | JSON array of playlists (managed via Web UI) |
@@ -371,11 +368,6 @@ Allstarr automatically fills your Spotify playlists (like Release Radar and Disc
# Enable the feature
SPOTIFY_IMPORT_ENABLED=true
# Sync window settings (used to avoid fetching too frequently)
SPOTIFY_IMPORT_SYNC_START_HOUR=16
SPOTIFY_IMPORT_SYNC_START_MINUTE=15
SPOTIFY_IMPORT_SYNC_WINDOW_HOURS=2
# Matching interval (24 hours = once per day)
SPOTIFY_IMPORT_MATCHING_INTERVAL_HOURS=24

7
allstarr/Controllers/AdminController.cs
View File

@@ -166,8 +166,7 @@ public class AdminController : ControllerBase
spotifyImport = new
{
enabled = _spotifyImportSettings.Enabled,
syncTime = $"{_spotifyImportSettings.SyncStartHour:D2}:{_spotifyImportSettings.SyncStartMinute:D2}",
syncWindowHours = _spotifyImportSettings.SyncWindowHours,
matchingIntervalHours = _spotifyImportSettings.MatchingIntervalHours,
playlistCount = _spotifyImportSettings.Playlists.Count
},
deezer = new
@@ -1392,9 +1391,7 @@ public class AdminController : ControllerBase
spotifyImport = new
{
enabled = _spotifyImportSettings.Enabled,
syncStartHour = _spotifyImportSettings.SyncStartHour,
syncStartMinute = _spotifyImportSettings.SyncStartMinute,
syncWindowHours = _spotifyImportSettings.SyncWindowHours,
matchingIntervalHours = _spotifyImportSettings.MatchingIntervalHours,
playlists = _spotifyImportSettings.Playlists.Select(p => new
{
name = p.Name,

21
allstarr/Models/Settings/SpotifyImportSettings.cs
View File

@@ -59,27 +59,6 @@ public class SpotifyImportSettings
/// </summary>
public bool Enabled { get; set; }
/// <summary>
/// Hour when Spotify Import plugin runs (24-hour format, 0-23)
/// NOTE: This setting is now optional and only used for the sync window check.
/// The fetcher will search backwards from current time for the last 48 hours,
/// so timezone confusion is avoided.
/// </summary>
public int SyncStartHour { get; set; } = 16;
/// <summary>
/// Minute when Spotify Import plugin runs (0-59)
/// NOTE: This setting is now optional and only used for the sync window check.
/// </summary>
public int SyncStartMinute { get; set; } = 15;
/// <summary>
/// How many hours to search for missing tracks files after sync start time
/// This prevents the fetcher from running too frequently.
/// Set to 0 to disable the sync window check and always search on startup.
/// </summary>
public int SyncWindowHours { get; set; } = 2;
/// <summary>
/// How often to run track matching in hours.
/// Spotify playlists like Discover Weekly update once per week, Release Radar updates weekly.

2
allstarr/Program.cs
View File

@@ -359,7 +359,7 @@ builder.Services.Configure<SpotifyImportSettings>(options =>
}
// Log configuration at startup
Console.WriteLine($"Spotify Import: Enabled={options.Enabled}, SyncHour={options.SyncStartHour}:{options.SyncStartMinute:D2}, WindowHours={options.SyncWindowHours}");
Console.WriteLine($"Spotify Import: Enabled={options.Enabled}, MatchingInterval={options.MatchingIntervalHours}h");
Console.WriteLine($"Spotify Import Playlists: {options.Playlists.Count} configured");
foreach (var playlist in options.Playlists)
{

213
allstarr/Services/Spotify/SpotifyMissingTracksFetcher.cs
View File

@@ -44,7 +44,7 @@ public class SpotifyMissingTracksFetcher : BackgroundService
public async Task TriggerFetchAsync()
{
_logger.LogInformation("Manual fetch triggered");
await FetchMissingTracksAsync(CancellationToken.None, bypassSyncWindowCheck: true);
await FetchMissingTracksAsync(CancellationToken.None);
}
protected override async Task ExecuteAsync(CancellationToken stoppingToken)
@@ -84,20 +84,7 @@ public class SpotifyMissingTracksFetcher : BackgroundService
_logger.LogInformation("Spotify Import ENABLED");
_logger.LogInformation("Configured Playlists: {Count}", _spotifySettings.Value.Playlists.Count);
// Log the search schedule
var settings = _spotifySettings.Value;
var syncTime = DateTime.Today
.AddHours(settings.SyncStartHour)
.AddMinutes(settings.SyncStartMinute);
var syncEndTime = syncTime.AddHours(settings.SyncWindowHours);
_logger.LogInformation("Search Schedule:");
_logger.LogInformation(" Plugin sync time: {Time:HH:mm} UTC (configured)", syncTime);
_logger.LogInformation(" Search window: {Start:HH:mm} - {End:HH:mm} UTC ({Hours}h window)",
syncTime, syncEndTime, settings.SyncWindowHours);
_logger.LogInformation(" Will search for new files once per day after sync window ends");
_logger.LogInformation(" Background check interval: 5 minutes");
_logger.LogInformation("Background check interval: 5 minutes");
// Fetch playlist names from Jellyfin
await LoadPlaylistNamesAsync();
@@ -109,7 +96,7 @@ public class SpotifyMissingTracksFetcher : BackgroundService
}
_logger.LogInformation("========================================");
// Check if we should run on startup
// Run on startup if we don't have cache
if (!_hasRunOnce)
{
var shouldRun = await ShouldRunOnStartupAsync();
@@ -118,7 +105,7 @@ public class SpotifyMissingTracksFetcher : BackgroundService
_logger.LogInformation("Running initial fetch on startup");
try
{
await FetchMissingTracksAsync(stoppingToken, bypassSyncWindowCheck: true);
await FetchMissingTracksAsync(stoppingToken);
_hasRunOnce = true;
}
catch (Exception ex)
@@ -128,21 +115,20 @@ public class SpotifyMissingTracksFetcher : BackgroundService
}
else
{
_logger.LogInformation("Skipping startup fetch - already have current files");
_logger.LogInformation("Skipping startup fetch - already have cached files");
_hasRunOnce = true;
}
}
// Background loop - check for new files every 5 minutes
while (!stoppingToken.IsCancellationRequested)
{
try
{
// Only fetch if we're past today's sync window AND we haven't fetched today yet
var shouldFetch = await ShouldFetchNowAsync();
if (shouldFetch)
{
await FetchMissingTracksAsync(stoppingToken);
_hasRunOnce = true;
}
}
catch (Exception ex)
@@ -156,42 +142,29 @@ public class SpotifyMissingTracksFetcher : BackgroundService
private async Task<bool> ShouldFetchNowAsync()
{
var settings = _spotifySettings.Value;
// Check if we have recent cache files (within last 24 hours)
var now = DateTime.UtcNow;
var cacheThreshold = now.AddHours(-24);
// Calculate today's sync window
var todaySync = now.Date
.AddHours(settings.SyncStartHour)
.AddMinutes(settings.SyncStartMinute);
var todaySyncEnd = todaySync.AddHours(settings.SyncWindowHours);
// Only fetch if we're past today's sync window
if (now < todaySyncEnd)
{
return false;
}
// Check if we already have today's files
foreach (var playlistName in _playlistIdToName.Values)
{
var filePath = GetCacheFilePath(playlistName);
if (File.Exists(filePath))
if (!File.Exists(filePath))
{
var fileTime = File.GetLastWriteTimeUtc(filePath);
// If file is from today's sync or later, we already have it
if (fileTime >= todaySync)
{
continue;
}
// Missing cache file for this playlist
return true;
}
// Missing today's file for this playlist
return true;
var fileTime = File.GetLastWriteTimeUtc(filePath);
if (fileTime < cacheThreshold)
{
// Cache file is older than 24 hours
return true;
}
}
// All playlists have today's files
// All playlists have recent cache files
return false;
}
@@ -210,120 +183,43 @@ public class SpotifyMissingTracksFetcher : BackgroundService
{
_logger.LogInformation("=== STARTUP CACHE CHECK ===");
var settings = _spotifySettings.Value;
var now = DateTime.UtcNow;
var allPlaylistsHaveCache = true;
// Calculate today's sync window
var todaySync = now.Date
.AddHours(settings.SyncStartHour)
.AddMinutes(settings.SyncStartMinute);
var todaySyncEnd = todaySync.AddHours(settings.SyncWindowHours);
_logger.LogInformation("Today's sync window: {Start:yyyy-MM-dd HH:mm} - {End:yyyy-MM-dd HH:mm} UTC",
todaySync, todaySyncEnd);
_logger.LogInformation("Current time: {Now:yyyy-MM-dd HH:mm} UTC", now);
// If we're still before today's sync window end, we should have yesterday's or today's file
// Don't search again until after today's sync window ends
if (now < todaySyncEnd)
foreach (var playlistName in _playlistIdToName.Values)
{
_logger.LogInformation("We're before today's sync window end - checking if we have recent cache...");
var filePath = GetCacheFilePath(playlistName);
var cacheKey = $"spotify:missing:{playlistName}";
var allPlaylistsHaveCache = true;
foreach (var playlistName in _playlistIdToName.Values)
// Check file cache
if (File.Exists(filePath))
{
var filePath = GetCacheFilePath(playlistName);
var cacheKey = $"spotify:missing:{playlistName}";
var fileAge = DateTime.UtcNow - File.GetLastWriteTimeUtc(filePath);
_logger.LogInformation(" {Playlist}: Found file cache (age: {Age:F1}h)", playlistName, fileAge.TotalHours);
// Check file cache
if (File.Exists(filePath))
// Load into Redis if not already there
if (!await _cache.ExistsAsync(cacheKey))
{
var fileAge = DateTime.UtcNow - File.GetLastWriteTimeUtc(filePath);
_logger.LogInformation(" {Playlist}: Found file cache (age: {Age:F1}h)", playlistName, fileAge.TotalHours);
// Load into Redis if not already there
if (!await _cache.ExistsAsync(cacheKey))
{
await LoadFromFileCache(playlistName);
}
continue;
await LoadFromFileCache(playlistName);
}
// Check Redis cache
if (await _cache.ExistsAsync(cacheKey))
{
_logger.LogInformation(" {Playlist}: Found in Redis cache", playlistName);
continue;
}
// No cache found for this playlist
_logger.LogInformation(" {Playlist}: No cache found", playlistName);
allPlaylistsHaveCache = false;
continue;
}
if (allPlaylistsHaveCache)
// Check Redis cache
if (await _cache.ExistsAsync(cacheKey))
{
_logger.LogInformation("=== ALL PLAYLISTS HAVE CACHE - SKIPPING STARTUP FETCH ===");
_logger.LogInformation("Will search again after {Time:yyyy-MM-dd HH:mm} UTC", todaySyncEnd);
return false;
_logger.LogInformation(" {Playlist}: Found in Redis cache", playlistName);
continue;
}
// No cache found for this playlist
_logger.LogInformation(" {Playlist}: No cache found", playlistName);
allPlaylistsHaveCache = false;
}
// If we're after today's sync window end, check if we already have today's file
if (now >= todaySyncEnd)
if (allPlaylistsHaveCache)
{
_logger.LogInformation("We're after today's sync window end - checking if we already fetched today's files...");
var allPlaylistsHaveTodaysFile = true;
foreach (var playlistName in _playlistIdToName.Values)
{
var filePath = GetCacheFilePath(playlistName);
var cacheKey = $"spotify:missing:{playlistName}";
// Check if file exists and was created today (after sync start)
if (File.Exists(filePath))
{
var fileTime = File.GetLastWriteTimeUtc(filePath);
// File should be from today's sync window or later
if (fileTime >= todaySync)
{
var fileAge = DateTime.UtcNow - fileTime;
_logger.LogInformation(" {Playlist}: Have today's file (created {Time:yyyy-MM-dd HH:mm}, age: {Age:F1}h)",
playlistName, fileTime, fileAge.TotalHours);
// Load into Redis if not already there
if (!await _cache.ExistsAsync(cacheKey))
{
await LoadFromFileCache(playlistName);
}
continue;
}
else
{
_logger.LogInformation(" {Playlist}: File is old (from {Time:yyyy-MM-dd HH:mm}, before today's sync)",
playlistName, fileTime);
}
}
else
{
_logger.LogInformation(" {Playlist}: No file found", playlistName);
}
allPlaylistsHaveTodaysFile = false;
}
if (allPlaylistsHaveTodaysFile)
{
_logger.LogInformation("=== ALL PLAYLISTS HAVE TODAY'S FILES - SKIPPING STARTUP FETCH ===");
// Calculate when to search next (tomorrow after sync window)
var tomorrowSyncEnd = todaySyncEnd.AddDays(1);
_logger.LogInformation("Will search again after {Time:yyyy-MM-dd HH:mm} UTC", tomorrowSyncEnd);
return false;
}
_logger.LogInformation("=== ALL PLAYLISTS HAVE CACHE - SKIPPING STARTUP FETCH ===");
return false;
}
_logger.LogInformation("=== WILL FETCH ON STARTUP ===");
@@ -380,32 +276,9 @@ public class SpotifyMissingTracksFetcher : BackgroundService
}
}
private async Task FetchMissingTracksAsync(CancellationToken cancellationToken, bool bypassSyncWindowCheck = false)
private async Task FetchMissingTracksAsync(CancellationToken cancellationToken)
{
var settings = _spotifySettings.Value;
var now = DateTime.UtcNow;
var syncStart = now.Date
.AddHours(settings.SyncStartHour)
.AddMinutes(settings.SyncStartMinute);
var syncEnd = syncStart.AddHours(settings.SyncWindowHours);
// Only run after the sync window has passed (unless bypassing for startup)
if (!bypassSyncWindowCheck && now < syncEnd)
{
_logger.LogInformation("Skipping fetch - sync window not passed yet (now: {Now}, window ends: {End})",
now, syncEnd);
return;
}
if (bypassSyncWindowCheck)
{
_logger.LogInformation("=== FETCHING MISSING TRACKS (STARTUP MODE) ===");
}
else
{
_logger.LogInformation("=== FETCHING MISSING TRACKS (SYNC WINDOW PASSED) ===");
}
_logger.LogInformation("=== FETCHING MISSING TRACKS ===");
_logger.LogInformation("Processing {Count} playlists", _playlistIdToName.Count);
// Track when we find files to optimize search for other playlists