Sign up

Jeremy Quinto

Spotify Listening History App #2 - Weekly Chart Generation

Spotify Listening History App #2 - Weekly Chart Generation

Creating weekly Billboard-style charts based on my Spotify listening history

Jeremy Quinto's photo
Jeremy Quinto
Oct 28, 2025·

13 min read

Hello! Welcome back to my personal blog. In this article, I’ll continue talking about my personal Spotify Listening History App, which I built on AWS. If you didn’t read the the previous article, going over the capabilities of the application, then you can read it here:

Part 1

Weekly Chart Generation

The initial idea that inspired me to create this app was to create an application that would generate weekly charts based on my listening history that mimic the Billboard Hot 100 and Billboard 200 charts. This article will be going over how I was able to implement this chart feature in my application.

(All UI is in progress😅)

Last article, we went over the data ingestion/aggregation pipeline. For ingestion, I used Eventbridge to schedule a lambda function to run every hour, and it would call the Spotify API to get my recent listening history, and add it to a DynamoDB recent-listening-history table. The TTL of every item in the recent-listening-history table is 4 weeks.

For weekly chart generation, I also used Eventbridge to schedule a lambda function, this time to run every Friday morning. This lambda function will fetch the recent listening history, aggregate the totals for each song, artist, and album, and generate the new weekly chart for each of them. The data for each chart is stored in an S3 bucket as JSON files, and the frontend uses an API endpoint to fetch the data to display.

Charting Algorithm

There are three charts generated each week:

  • “My Hot 100” - top 100 songs

  • “My Artists 25” - top 25 artists

  • “My Albums 50” - top 50 albums

Now I do listen to a lot of music, but there are a lot of weeks where I don’t listen to 100 songs, 25 artists, or 50 different albums over a one week span, so the weekly charts don’t just pull the listening history from the last week.

When fetching the listening history to generate the charts, I actually fetch the listening history from the last 3 weeks, and weight the most recent week with more points than the previous week (and the second most previous week more than the 3rd most previous week). Typically, this is enough to fill out all three of the charts. This is also why the TTL of the recent-listening-history table is 4 weeks, as the three most recent weeks need to be fetched for every chart generation, and an extra week of padding is added in case errors occur.

Here is an example of the code for calculate the chart points for the song chart:

import {
  CurrentSongChartPointData,
  ListeningHistoryDynamoDBItem,
} from "../types";

// Calculate chart points based on listening history over the last three weeks
export const calculateSongChartPointsFromListeningHistory = (
  listeningHistory: ListeningHistoryDynamoDBItem[]
): CurrentSongChartPointData[] => {
  const songPoints = new Map<string, number>();
  const songDetails = new Map<
    string,
    Omit<CurrentSongChartPointData, "points">
  >();

  const now = new Date();
  const oneWeekAgo = new Date(now.getTime() - 7 * 24 * 60 * 60 * 1000);
  const twoWeeksAgo = new Date(now.getTime() - 14 * 24 * 60 * 60 * 1000);

  for (const item of listeningHistory) {
    const playedAt = new Date(item.timestamp);
    let points = 0;

    if (playedAt >= oneWeekAgo) {
      // Last week
      points = 5;
    } else if (playedAt >= twoWeeksAgo) {
      // Week before
      points = 2;
    } else {
      // Week before that
      points = 1;
    }

    const currentPoints = songPoints.get(item.track_id) || 0;
    songPoints.set(item.track_id, currentPoints + points);

    // Store song details if not already present
    if (!songDetails.has(item.track_id)) {
      songDetails.set(item.track_id, {
        track_id: item.track_id,
        track_name: item.track_name,
        artist_name: item.artist_name,
        album_name: item.album_name,
        artist_id: item.artist_id!,
        album_id: item.album_id!,
        album_cover_url: item.album_cover_url!,
        position: 0, // Placeholder, will be set later
        genre: item.genre,
      });
    }
  }

  const aggregatedData: CurrentSongChartPointData[] = [];
  for (const [trackId, points] of songPoints.entries()) {
    const details = songDetails.get(trackId);
    if (details) {
      aggregatedData.push({
        ...details,
        points: points,
      });
    }
  }

  // Sort the aggregated data in descending order by points to rank the songs
  aggregatedData.sort((a, b) => b.points - a.points);

  for (let i = 0; i < aggregatedData.length; i++) {
    aggregatedData[i].position = i + 1;
  }

  return aggregatedData;
};

After all three charts are calculated, the results are uploaded to an S3 bucket as JSON files, and the metadata for each generated chart (the key name, timestamp, etc) are stored in a separate DynamoDB table, allowing my API to easily query and figure out what chart exists.

Chart Stats

One thing that separates these Billboard-style charts from just looking at a page that shows recent listening history - and the main reason why I wanted to create an app with these Billboard-style charts - are the chart statistics and chart history that come with them. These are things like peak position, last week’s position, and weeks on chart. These give a unique insight into my listening history that are fun to explore.

I actually do have pages that show raw stats for listening history on a song/artist/album basis, but the charts are more fun!

All chart stats (peak, last_week_position, weeks_charted, etc) are stored in a separate DynamoDB table, containing historical chart data for each song. The logic for updating these items is handled by the weekly chart generation lambda function, and is quite complicated. But here’s a snippet for the song chart data entries for now:

export const getAndUpdateChartEntry = async (
  entry: CurrentSongChartPointData,
  position: number,
  recent_play_count: number,
  chart_timestamp: string
): Promise<SongChartData> => {
  // Step 1. Get song data from DynamoDB
  const getItemParams: GetItemCommandInput = {
    TableName: SONG_HISTORY_TABLE_NAME,
    Key: {
      artist_id: { S: entry.artist_id },
      track_id: { S: entry.track_id },
    },
  };

  const { Item: existingItem } = await DYNAMODB_CLIENT.send(
    new GetItemCommand(getItemParams)
  );

  let playCount = recent_play_count;
  let lastWeekPosition: number | null = null;
  let weeksOnChart = 1;
  let peakPosition = position;

  if (existingItem) {
    // If the song exists, calculate the new metrics
    const existingPlayCount = existingItem.play_count?.N
      ? parseInt(existingItem.play_count.N)
      : 0;
    // Get the current position from database - this becomes last week's position
    const existingCurrentPosition = existingItem.position?.N
      ? parseInt(existingItem.position.N)
      : null;
    const existingWeeksOnChart = existingItem.weeks_on_chart?.N
      ? parseInt(existingItem.weeks_on_chart.N)
      : 0;
    const existingPeakPosition = existingItem.peak_position?.N
      ? parseInt(existingItem.peak_position.N)
      : position;

    playCount += existingPlayCount;
    lastWeekPosition = existingCurrentPosition;
    weeksOnChart = existingWeeksOnChart + 1;
    peakPosition = Math.min(existingPeakPosition, position);
  }

  // Determine if this song should be marked as "charted" (position <= 100)
  const isCharted = position <= 100;

  // Step 3. Update DynamoDB with new song data
  let updateExpression =
    "SET #play_count = :total_plays, " +
    "#peak_position = :peak_position, " +
    "#weeks_on_chart = :weeks_on_chart, " +
    "#last_charted_at = :last_charted_at, " +
    "#track_name = if_not_exists(#track_name, :track_name), " +
    "#artist_name = if_not_exists(#artist_name, :artist_name), " +
    "#album_name = if_not_exists(#album_name, :album_name), " +
    "#album_cover_url = if_not_exists(#album_cover_url, :album_cover_url)";

  // Only set position if the song is in the top 100
  if (isCharted) {
    updateExpression += ", #position = :current_position";
  }

  if (entry.album_id) {
    updateExpression += ", #album_id = if_not_exists(#album_id, :album_id)";
  }

  // Only set last_week_position if we have a value for it
  if (lastWeekPosition !== null) {
    updateExpression += ", #last_week_position = :last_week_position";
  }

  if (entry.genre) {
    updateExpression += ", #genre = if_not_exists(#genre, :genre)";
  }

  const expressionAttributeNames: Record<string, string> = {
    "#play_count": "play_count",
    "#peak_position": "peak_position",
    "#weeks_on_chart": "weeks_on_chart",
    "#last_charted_at": "last_charted_at",
    "#track_name": "track_name",
    "#artist_name": "artist_name",
    "#album_name": "album_name",
    "#album_cover_url": "album_cover_url",
  };

  const expressionAttributeValues: Record<string, any> = {
    ":total_plays": { N: playCount.toString() },
    ":peak_position": { N: peakPosition.toString() },
    ":weeks_on_chart": { N: weeksOnChart.toString() },
    ":last_charted_at": { S: chart_timestamp },
    ":track_name": { S: entry.track_name },
    ":artist_name": { S: entry.artist_name },
    ":album_name": { S: entry.album_name },
    ":album_cover_url": { S: entry.album_cover_url },
  };

  // Only add position-related attributes if the song is in the top 100
  if (isCharted) {
    expressionAttributeNames["#position"] = "position";
    expressionAttributeValues[":current_position"] = { N: position.toString() };
  }

  if (entry.album_id) {
    expressionAttributeNames["#album_id"] = "album_id";
    expressionAttributeValues[":album_id"] = { S: entry.album_id };
  }

  if (lastWeekPosition !== null) {
    expressionAttributeNames["#last_week_position"] = "last_week_position";
    expressionAttributeValues[":last_week_position"] = {
      N: lastWeekPosition.toString(),
    };
  }

  if (entry.genre) {
    expressionAttributeNames["#genre"] = "genre";
    expressionAttributeValues[":genre"] = { S: entry.genre };
  }

  const updateItemParams: UpdateItemCommandInput = {
    TableName: SONG_HISTORY_TABLE_NAME,
    Key: {
      artist_id: { S: entry.artist_id },
      track_id: { S: entry.track_id },
    },
    UpdateExpression: updateExpression,
    ExpressionAttributeNames: expressionAttributeNames,
    ExpressionAttributeValues: expressionAttributeValues,
  };

  await DYNAMODB_CLIENT.send(new UpdateItemCommand(updateItemParams));

  // Step 4. Return chart entry
  return {
    position: position,
    track_id: entry.track_id,
    track_name: entry.track_name,
    peak: peakPosition,
    last_week: lastWeekPosition,
    weeks_on_chart: weeksOnChart,
    artist_name: entry.artist_name,
    artist_id: entry.artist_id,
    album_id: entry.album_id,
    album_name: entry.album_name,
    album_cover: entry.album_cover_url,
    plays_since_last_week: recent_play_count,
    points: entry.points,
    genre: entry.genre,
    is_debut: weeksOnChart === 1,
  };
};

You can see for charting songs, the weeks_on_chart is obviously incremented by 1. The last_week_position is easy to derive, as well as the peak (peak position). Very similar logic is applied for the artist and album charts.

In the future I plan on adding more insights, like:

  • Number of Top 10 hits for an artist/album

  • Number of #1 hits for an artist/album

  • Peak number of songs on the Hot 100 chart for artist/album

  • Weeks at number 1, weeks in Top 10

Also, looking at the code over again, it is probably much more optimal to use BatchGetItem and BatchWriteItem commands… get all items at once, process their stats, and then write them in batches. Oops!

Uploading to S3

Below is just the basic code for uploading a chart to S3: 

import {
  S3Client,
  PutObjectCommand,
  PutObjectCommandInput,
} from "@aws-sdk/client-s3";
import { ChartFile } from "../types";

const S3_CLIENT = new S3Client({});
const { SONG_CHART_HISTORY_BUCKET_NAME } = process.env;

export const uploadChart = async (
  chartFile: ChartFile,
  key: string
): Promise<void> => {
  const body = chartFile;

  const putObjectParams: PutObjectCommandInput = {
    Bucket: SONG_CHART_HISTORY_BUCKET_NAME,
    Key: key,
    Body: JSON.stringify(body, null, 2),
    ContentType: "application/json",
  };

  await S3_CLIENT.send(new PutObjectCommand(putObjectParams));
};

Playlist Generation

After the chart generation runs every Friday, I can go to my website and see what the new charts look like. I do plan on adding a basic notification system to notify me when new charts are ready, but in the meantime, I’ve added something different - an automatic playlist manager for my personal Hot 100.

Right after the chart generation lambda function runs, a separate lambda function runs that will get the most recently generated Hot 100 chart, and update a playlist that I have in Spotify.

It first obtains the most recent chart from S3. Then, it tries to either get or create the playlist by name (in case it was deleted):

const SPOTIFY_API_BASE = "https://api.spotify.com/v1";

/**
 * Search for a playlist by name in the current user's playlists
 * @param playlistName - Name of the playlist to search for
 * @returns The playlist ID if found, null otherwise
 */
export const getPlaylist = async (
  playlistName: string,
  accessToken: string
): Promise<string | null> => {
  console.log(`Fetching playlist: ${playlistName}`);

  try {
    // Get current user's playlists (with pagination)
    let nextUrl = `${SPOTIFY_API_BASE}/me/playlists?limit=50`;

    while (nextUrl) {
      const response = await fetch(nextUrl, {
        headers: {
          Authorization: `Bearer ${accessToken}`,
          "Content-Type": "application/json",
        },
      });

      if (!response.ok) {
        throw new Error(
          `Failed to fetch playlists: ${response.status} ${response.statusText}`
        );
      }

      const data = await response.json();

      // Search for playlist by name (case-insensitive)
      const existingPlaylist = data.items.find(
        (playlist: any) =>
          playlist.name.toLowerCase() === playlistName.toLowerCase()
      );

      if (existingPlaylist) {
        console.log(
          `Playlist "${playlistName}" found with ID: ${existingPlaylist.id}`
        );
        return existingPlaylist.id;
      }

      nextUrl = data.next;
    }

    console.log(`Playlist "${playlistName}" not found`);
    return null;
  } catch (error) {
    console.error("Error fetching playlist:", error);
    throw error;
  }
};

/**
 * Create a new playlist for the current user
 * @param playlistName - Name of the new playlist
 * @returns The created playlist ID
 */
export const createPlaylist = async (
  playlistName: string,
  accessToken: string
): Promise<string> => {
  console.log(`Creating playlist: ${playlistName}`);

  try {
    // First, get the current user's ID
    const userResponse = await fetch(`${SPOTIFY_API_BASE}/me`, {
      headers: {
        Authorization: `Bearer ${accessToken}`,
        "Content-Type": "application/json",
      },
    });

    if (!userResponse.ok) {
      throw new Error(
        `Failed to get user info: ${userResponse.status} ${userResponse.statusText}`
      );
    }

    const userData = await userResponse.json();
    const userId = userData.id;

    // Create the playlist
    const createResponse = await fetch(
      `${SPOTIFY_API_BASE}/users/${userId}/playlists`,
      {
        method: "POST",
        headers: {
          Authorization: `Bearer ${accessToken}`,
          "Content-Type": "application/json",
        },
        body: JSON.stringify({
          name: playlistName,
          description: "My Current Hot 100 Playlist!",
          public: false,
        }),
      }
    );

    if (!createResponse.ok) {
      throw new Error(
        `Failed to create playlist: ${createResponse.status} ${createResponse.statusText}`
      );
    }

    const newPlaylist = await createResponse.json();
    console.log(
      `Playlist "${playlistName}" created successfully with ID: ${newPlaylist.id}`
    );
    return newPlaylist.id;
  } catch (error) {
    console.error("Error creating playlist:", error);
    throw error;
  }
};

/**
 * Get or create a playlist - checks if playlist exists, creates it if not
 * @param playlistName - Name of the playlist
 * @returns The playlist ID
 */
export const getOrCreatePlaylist = async (
  playlistName: string,
  accessToken: string
): Promise<string> => {
  try {
    // First try to get existing playlist
    let playlistId = await getPlaylist(playlistName, accessToken);

    // If not found, create it
    if (!playlistId) {
      playlistId = await createPlaylist(playlistName, accessToken);
    }

    return playlistId;
  } catch (error) {
    console.error("Error in getOrCreatePlaylist:", error);
    throw error;
  }
};

Next, it will update the playlist with the new chart data:

import { SongChartEntry } from "../types";

const SPOTIFY_API_BASE = "https://api.spotify.com/v1";

/**
 * Remove all tracks from a playlist
 * @param playlistId - Spotify playlist ID
 */
const clearPlaylist = async (
  playlistId: string,
  accessToken: string
): Promise<void> => {
  try {
    console.log(`Clearing all tracks from playlist ${playlistId}...`);

    // Get all tracks in the playlist
    let allTracks: any[] = [];
    let nextUrl = `${SPOTIFY_API_BASE}/playlists/${playlistId}/tracks?limit=100`;

    while (nextUrl) {
      const response = await fetch(nextUrl, {
        headers: {
          Authorization: `Bearer ${accessToken}`,
          "Content-Type": "application/json",
        },
      });

      if (!response.ok) {
        throw new Error(
          `Failed to fetch playlist tracks: ${response.status} ${response.statusText}`
        );
      }

      const data = await response.json();
      allTracks = [...allTracks, ...data.items];
      nextUrl = data.next;
    }

    if (allTracks.length === 0) {
      console.log("Playlist is already empty");
      return;
    }

    // Remove tracks in batches of 100 (Spotify API limit)
    const batchSize = 100;
    for (let i = 0; i < allTracks.length; i += batchSize) {
      const batch = allTracks.slice(i, i + batchSize);
      const tracksToRemove = batch.map((item) => ({
        uri: item.track.uri,
      }));

      const response = await fetch(
        `${SPOTIFY_API_BASE}/playlists/${playlistId}/tracks`,
        {
          method: "DELETE",
          headers: {
            Authorization: `Bearer ${accessToken}`,
            "Content-Type": "application/json",
          },
          body: JSON.stringify({
            tracks: tracksToRemove,
          }),
        }
      );

      if (!response.ok) {
        throw new Error(
          `Failed to remove tracks: ${response.status} ${response.statusText}`
        );
      }
    }

    console.log(`Removed ${allTracks.length} tracks from playlist`);
  } catch (error) {
    console.error("Error clearing playlist:", error);
    throw error;
  }
};

/**
 * Add tracks to a playlist
 * @param playlistId - Spotify playlist ID
 * @param trackUris - Array of Spotify track URIs
 */
const addTracksToPlaylist = async (
  playlistId: string,
  trackUris: string[],
  accessToken: string
): Promise<void> => {
  try {
    console.log(
      `Adding ${trackUris.length} tracks to playlist ${playlistId}...`
    );

    // Add tracks in batches of 100 (Spotify API limit)
    const batchSize = 100;
    for (let i = 0; i < trackUris.length; i += batchSize) {
      const batch = trackUris.slice(i, i + batchSize);

      const response = await fetch(
        `${SPOTIFY_API_BASE}/playlists/${playlistId}/tracks`,
        {
          method: "POST",
          headers: {
            Authorization: `Bearer ${accessToken}`,
            "Content-Type": "application/json",
          },
          body: JSON.stringify({
            uris: batch,
          }),
        }
      );

      if (!response.ok) {
        throw new Error(
          `Failed to add tracks: ${response.status} ${response.statusText}`
        );
      }
    }

    console.log(`Successfully added ${trackUris.length} tracks to playlist`);
  } catch (error) {
    console.error("Error adding tracks to playlist:", error);
    throw error;
  }
};

/**
 * Update playlist description with timestamp
 * @param playlistId - Spotify playlist ID
 * @param accessToken - Spotify access token
 */
const updatePlaylistDescription = async (
  playlistId: string,
  accessToken: string
): Promise<void> => {
  try {
    const currentTimestamp = new Date().toLocaleDateString("en-US", {
      weekday: "long",
      year: "numeric",
      month: "long",
      day: "numeric",
    });
    const description = `My Current Hot 100 Playlist! Last Updated ${currentTimestamp}`;

    console.log(
      `Updating playlist description with timestamp: ${currentTimestamp}`
    );

    const response = await fetch(
      `${SPOTIFY_API_BASE}/playlists/${playlistId}`,
      {
        method: "PUT",
        headers: {
          Authorization: `Bearer ${accessToken}`,
          "Content-Type": "application/json",
        },
        body: JSON.stringify({
          description: description,
        }),
      }
    );

    if (!response.ok) {
      throw new Error(
        `Failed to update playlist description: ${response.status} ${response.statusText}`
      );
    }

    console.log("Playlist description updated successfully");
  } catch (error) {
    console.error("Error updating playlist description:", error);
    throw error;
  }
};

/**
 * Generate playlist from chart entries - clears existing tracks and adds new ones in chart order
 * @param playlistId - Spotify playlist ID
 * @param chartEntries - Array of song chart entries sorted by position
 * @returns Object with results of the operation
 */
export const generatePlaylist = async (
  playlistId: string,
  chartEntries: SongChartEntry[],
  accessToken: string
): Promise<{
  tracksAdded: number;
  totalEntries: number;
}> => {
  try {
    console.log(
      `Generating playlist ${playlistId} with ${chartEntries.length} chart entries...`
    );

    // Step 1: Clear existing playlist
    await clearPlaylist(playlistId, accessToken);

    // Step 2: Sort chart entries by position to ensure correct order
    const sortedEntries = [...chartEntries].sort(
      (a, b) => a.position - b.position
    );

    // Step 3: Convert track IDs to URIs
    const trackUris: string[] = sortedEntries.map(
      (entry) => `spotify:track:${entry.track_id}`
    );

    console.log(`Converting ${sortedEntries.length} track IDs to URIs...`);

    // Step 4: Add tracks to playlist in order
    if (trackUris.length > 0) {
      await addTracksToPlaylist(playlistId, trackUris, accessToken);
    }

    // Step 5: Update playlist description with current timestamp
    await updatePlaylistDescription(playlistId, accessToken);

    const result = {
      tracksAdded: trackUris.length,
      totalEntries: chartEntries.length,
    };

    console.log(
      `Playlist generation complete: ${result.tracksAdded} tracks added from ${result.totalEntries} chart entries`
    );

    return result;
  } catch (error) {
    console.error("Error generating playlist:", error);
    throw error;
  }
};

This handles clearing the playlist, updating the playlist with the new tracks, and then updating the description all in one go!

Chart Generation Done!

That’s all for this article. In the next article, we’ll go over the different image processing aspects that I implemented to create the cool song/album banners as shown below:

Thanks for reading!

AWS