# Pastebin wxHSJI2G
# Personal Information
#### Name: Vardan Saini
#### School: University of Alberta
#### Github: https://github.com/vardansaini
#### Linkedin: https://www.linkedin.com/in/vardansaini/
### Portfolio: https://vardansaini.github.io/vardansaini/
#### IRC nickname: vscode
#### email: vardan1@ualberta.ca

# Project Overview
By linking their Apple Music account with ListenBrainz, individuals are able to use ListenBrainz to play their music selections. The integration into BrainzPlayer allows for this capability. By utilizing a content resolver, we can match the MusicBrainz Identifier (MBID) to the Apple Music catalog, permitting users to transfer playlists to their Apple Music account.

# Introduction

ListenBrainz currently supports integration with Spotify, but there is a growing demand for integration with other

music services. Apple Music is a popular music streaming service, and integrating it into ListenBrainz will provide

more options for users to access personalized recommendations. This proposal outlines the steps to integrate Apple

Music into ListenBrainz.

  

## OAuth Integration

  

### Frontend

  

1. The first step in integrating Apple Music into ListenBrainz is to configure OAuth on the frontend. The MusicKit

JavaScript library can be used to authenticate users with their Apple Music accounts using the OAuth protocol.

```html

<script src="https://js-cdn.music.apple.com/musickit/v3/musickit.js" data-web-components async></script>

```

2. The developer token needs to be generated on the backend and sent to the frontend. Once the MusicKit library is

included in the web page, the `configure` method is called to configure an instance of MusicKit on the Web.

```javascript

document.addEventListener('musickitloaded', async function () {

try {

await MusicKit.configure({

developerToken: 'DEVELOPER-TOKEN',

app: {

name: 'ListenBrainz',

build: 'GIT-COMMIT-SHA',

},

});

} catch (err) {

// Handle configuration error

}

const music = MusicKit.getInstance();

});

```

3. The `authorize` method can be used to authenticate the user and access their Apple Music account. This step involves

the following:

```javascript

music.authorize().then(musicUserToken => {

submitMusicUserTokenToLB(musicUserToken);

});

```

  

### Backend

  

1. The music services callback URL (`/music-services/{service}/callback`) receives the authorization code in case of

of other music services. However, in the case of Apple Music there is no client secret oauth available. Instead we

do OAuth client side using MusicKit and once the user has granted access, the Music-User-Token is sent to server at

callback URL.

  

2. Apple Music's catalog varies with the user's storefront so store we should retrieve and store it.

```python

def get_storefront(music_user_token):

headers = {'Authorization': f'Bearer {music_user_token}'}

response = requests.get('https://api.music.apple.com/v1/me/storefront', headers=headers)

return response.json()['data'][0]['id']

```

  

3. The token and storefront can be stored in the database to be used later.

```python

@profile_bp.route('/music-services/apple/callback')

@login_required

def apple_callback():

service = AppleService()

music_user_token = request.args.get('music_user_token')

service.get_storefront(music_user_token)

service.add_new_user(current_user.id, music_user_token, storefront)

return redirect(url_for('index'))

```

  

## Integrate Apple Music in BrainzPlayer

The BrainzPlayer is a web-based player used to play music on ListenBrainz from various services. It already supports

playing music from [Spotify](https://github.com/metabrainz/listenbrainz-server/blob/master/frontend/js/src/brainzplayer/SpotifyPlayer.tsx), [Youtube](https://github.com/metabrainz/listenbrainz-server/blob/master/frontend/js/src/brainzplayer/YoutubePlayer.tsx) and [Soundcloud](https://github.com/metabrainz/listenbrainz-server/blob/master/frontend/js/src/brainzplayer/SoundcloudPlayer.tsx).

  

1. MusicKit can be loaded on the page in the same as shown in the OAuth Frontend section.

2. Once the player is loaded, we can use [`MusicKitInstance.{play,pause}`](https://js-cdn.music.apple.com/musickit/v3/docs/index.html?path=/docs/reference-javascript-musickit-instance--page#pause)

etc. controls and add callbacks to `playbackStateWillChange` and `playbackStateDidChange` events to

integrate with BrainzPlayer controls.

  

```javascript

music.addEventListener('playbackStateWillChange', ({ oldState, state }) => {

console.log(`About to change the playback state from ${oldState} to ${state}`);

});

```

Before adding the event callbacks, we should check whether the user has authorized Apple Music.

3. Tracks can be enqueued in the MusicKit player using `setQueue` or `playNext` method and its Apple Music Identifier:

```javascript

await music.playNext({ song: '1561837008' });

```

4. If the listen does not have an Apple Music Identifier in it, we can query the search API as follows:

```javascript

const results = await music.api.search(term, {

types: 'songs',

limit: 10,

storefront: 'us'

});

```

Then we can enqueue the desired track from the search results in the player and play it.

  

## Build Content Resolver

  

The metadata content resolver is used to store metadata of the entire catalog present in an external service's database.

Currently, ListenBrainz has such a resolver for Spotify. As a part of this project, I intend to add a similar content

resolver for Apple Music as well.

  

1. Use the Apple Music API to get all storefronts (https://api.music.apple.com/v1/storefronts)

  

```json5

[

{

"id": "dz",

"type": "storefronts",

"href": "/v1/storefronts/dz",

"attributes": {

"explicitContentPolicy": "opt-in",

"name": "Algeria",

"defaultLanguageTag": "en-GB",

"supportedLanguageTags": [

"en-GB",

"fr-FR",

"ar"

]

}

},

// more storefronts ....

]

```

  

2. Get catalog charts for each storefront to seed metadata resolver (https://api.music.apple.com/v1/catalog/us/charts?types=songs,albums)

```json5

{

"results": {

"songs": [

{

"chart": "most-played",

"name": "Top Songs",

"orderId": "most-played:songs",

"next": "/v1/catalog/us/charts?chart=most-played&genre=20&limit=1&offset=1&types=songs",

"data": [

{

"id": "635016640",

"type": "songs",

"href": "/v1/catalog/us/songs/635016640",

"attributes": {

"albumName": "I Love You.",

"composerName": "Jesse, Zachary Abels & Jeremiah Freedman",

"playParams": {

"id": "635016640",

"kind": "song"

},

"name": "Sweater Weather",

"artistName": "The Neighbourhood",

// ... more fields

}

}

],

"href": "/v1/catalog/us/charts?chart=most-played&genre=20&limit=1&types=songs"

}

],

"albums": [

{

"chart": "most-played",

"name": "Top Albums",

"orderId": "most-played:albums",

"next": "/v1/catalog/us/charts?chart=most-played&genre=20&limit=1&offset=1&types=albums",

"data": [

{

"id": "1608118564",

"type": "albums",

"href": "/v1/catalog/us/albums/1608118564",

"attributes": {

"playParams": {

"id": "1608118564",

"kind": "album"

},

"name": "mainstream sellout",

"artistName": "Machine Gun Kelly",

// .. more fields

}

}

],

"href": "/v1/catalog/us/charts?chart=most-played&genre=20&limit=1&types=albums"

}

]

}

}

```

Using the ids from this data, seed the content resolver.

  

3. Get all albums of the artist (https://api.music.apple.com/v1/catalog/us/artists/{artist_id}/albums)

  

```json5

[

{

"id": "1482041821",

"type": "albums",

"href": "/v1/catalog/us/albums/1482041821",

"attributes": {

"copyright": "℗ 2019 Mom+Pop",

"genreNames": [

"Alternative",

"Music"

],

"releaseDate": "2020-02-14",

"upc": "858275058666",

"artwork": {

"width": 3000,

"height": 3000,

"url": "https://is3-ssl.mzstatic.com/image/thumb/Music125/v4/0b/b2/52/0bb2524d-ecfc-1bae-9c1e-218c978d7072/Honeymoon_3K.jpg/{w}x{h}bb.jpg",

"bgColor": "fffaa9",

"textColor1": "030005",

"textColor2": "363240",

"textColor3": "353226",

"textColor4": "5e5a55"

},

"url": "https://music.apple.com/us/album/honeymoon/1482041821",

"playParams": {

"id": "1482041821",

"kind": "album"

},

"recordLabel": "Mom+Pop",

"trackCount": 9,

"name": "Honeymoon",

"artistName": "Beach Bunny"

}

},

// .. more albums

]

```

4. Get all tracks of multiple albums (https://api.music.apple.com/v1/catalog/us/albums?ids={album_ids})

```json5

{

"data": [

{

"id": "1616728060",

"type": "albums",

"href": "/v1/catalog/us/albums/1616728060",

"attributes": {

"name": "Heart On My Sleeve",

"contentRating": "explicit",

"artistName": "Ella Mai"

},

"relationships": {

"tracks": {

"href": "/v1/catalog/us/albums/1616728060/tracks",

"data": [

{

"id": "1616728064",

"type": "songs",

"href": "/v1/catalog/us/songs/1616728064",

"attributes": {

"albumName": "Heart On My Sleeve",

"genreNames": [

"R&B/Soul",

"Music"

],

"trackNumber": 1,

"durationInMillis": 197314,

"releaseDate": "2022-05-06",

"isrc": "USUM72203653",

"artwork": {

"width": 3000,

"height": 3000,

"url": "https://is3-ssl.mzstatic.com/image/thumb/Music112/v4/03/45/19/034519dc-9ff9-7f63-3d02-23a101a0cc3a/22UMGIM01299.rgb.jpg/{w}x{h}bb.jpg",

"bgColor": "0b1b18",

"textColor1": "fc977a",

"textColor2": "ef5429",

"textColor3": "cc7e66",

"textColor4": "c14925"

},

"composerName": "Charles Hinshaw Jr, Dijon McFarlane, Ella Mai Howell, NICHOLAS MATTHEW BALDING, Peter Johnson & Shah Rukh Zaman Khan",

"playParams": {

"id": "1616728064",

"kind": "song"

},

"url": "https://music.apple.com/us/album/trying/1616728060?i=1616728064",

"discNumber": 1,

"artistName": "Ella Mai"

}

},

// ... more tracks

]

}

}

},

// ... more albums

]

}

```

5. Store data in a normalized tables in a database [schema](https://github.com/metabrainz/listenbrainz-server/blob/b28e14f2843f221cd5cbe555c9eabdc50c1a6e36/admin/timescale/create_tables.sql#L143-L190) similar to spotify content resolver and set a cache expiry.

6. Before retrieving data from API, check if it exists already in database and update cache accordingly.

# Timeline 

Week 1-4: Integrating Apple Music in BrainzPlayer (without OAuth), setting up controls, searching tracks and playing previews

Week 5-7: Getting Music User Token, storefront, storing it in database

Week 8-11: Build the content resolver to store Apple Music catalog metadata in ListenBrainz

Week 12-13: Buffer Period

# Other Information

### Tell us about the computer(s) you have available for working on your SoC project!
I have a Macbook pro 16 inch with 2.3 GHz 8 core intel i9 (9th gen)
, AMD Radeon Pro 5500M 4 GB, 16 GB 2667 MHz DDR4 and 1 TB SSD.

### When did you first start programming?
Since 2011, I've been involved in programming using Java, and over time, I've had the opportunity to work on numerous projects.

### What type of music do you listen to? (Please list a series of MBIDs as examples.)
I listen to Punajbi, English and Hindi music.
Here are MBID's of some of my favourite artists:-

Name: [Karan Aujla](https://musicbrainz.org/artist/4a779683-5404-4b90-a0d7-242495158265 "Aujla, Karan")
[MBID](https://musicbrainz.org/doc/MusicBrainz_Identifier): `4a779683-5404-4b90-a0d7-242495158265`

Name: [Sam Smith](https://musicbrainz.org/artist/5a85c140-dcf9-4dd2-b2c8-aff0471549f3 "Smith, Sam (English singer and songwriter)") (English singer and songwriter)
[MBID](https://musicbrainz.org/doc/MusicBrainz_Identifier): `5a85c140-dcf9-4dd2-b2c8-aff0471549f3`

Name: [Imagine Dragons](https://musicbrainz.org/artist/012151a8-0f9a-44c9-997f-ebd68b5389f9 "Imagine Dragons")
[MBID](https://musicbrainz.org/doc/MusicBrainz_Identifier): `012151a8-0f9a-44c9-997f-ebd68b5389f9`

### What aspects of the project you're applying for (e.g., MusicBrainz, AcousticBrainz, etc.) interest you the most?
Music is an essential part of my life, and I listen to it while doing everything. It is particularly crucial for me when I work out at the gym, which I do daily. As a software engineer who works on computer allot, I rely on Apple Music, and I wanted to integrate it with Listenbrainz, which supports various other platforms. Therefore, this project is a perfect blend of my favorite things.

### Have you ever used MusicBrainz to tag your files?
Not much but I have heard allot about it from my friends at univeristy.

### Have you contributed to other Open Source projects? If so, which projects and can we see some of your code?
I have previously made contributions to ListenBrainz, as well as to an open-source project called Capstone Dashboard which is available on GitHub at [https://github.com/open-uofa/capstone-course-dashboard](https://github.com/open-uofa/capstone-course-dashboard).

### What sorts of programming projects have you done on your own time?
I am conducting NLP research utilizing transformers, and have also applied my knowledge in PCGML and unity to develop a no-code platform for creating 2D games without programming. In addition, I have completed contract work involving MERN techstack to deliver websites, created my own portfolio using React, and have dabbled in Flutter to develop apps for leisure. Additionally, I have  experimented with Flask. For more examples of my work, please visit my Github profile: https://github.com/vardansaini