DJs often spend disproportionate amounts of time searching for music tracks, listening to them, and classifying them. How can we make this process quicker and more efficient?
We employ supervised machine learning techniques to analyze music tracks and automatically classify them into different playlists.
Data Collection and Feature Extraction
We utilize Librosa, NumPy, MatPlotLib, LyricsGenius, OpenAI, TinyTag, Pandas, Ast, NLTK, and Plotly to extract audio and textual features, conduct dimensionality reductions, and visualize distinctions.
Audio Features are first extracted using Librosa and NumPy. We extract Mel-Frequency Cepstral Coefficients, Zero Cross Rates, Root Mean Squares, Spectral Rolloffs, and Chromagram values across each 10 second chunk of an audio sample, and also store their means and variances. 
We use TinyTag to extract metadata from download files. We use the metadata to obtain lyrics for each song through the LyricsGenius API. 
The lyrics are cleaned, with stopwords and irrelevant punctuation removed. 
The lyrics are vectorized using the OpenAI Large Embeddings model.
A final data frame is created and stored.
Audio Exploratory Data Analysis
We utilize NumPy, MatPlotLib, Pandas, NLTK, Seaborn, and Plotly to conduct dimensionality reduction, analyze central tendencies, and visualize distinctions.
We calculate central tendencies of key audio features to identify differences. 
Kernel Density Estimate graphs are plotted for key audio features and separated by playlist to visualize differences in densities. 
Correlation heat maps are plotted to identify covariances between key audio features to inform feature engineering.
A principal component analysis is conducted and visualized to observe distinctions between playlists in terms of audio features.
Textual Exploratory Data Analysis
We utilize NumPy, MatPlotLib, Pandas, NLTK, Seaborn, and Plotly to conduct dimensionality reduction, analyze central tendencies, and visualize distinctions.
A t-SNE is conducted on 3 principal components on the lyrical features of the songs. These are visualized to identify differences across playlists. Distinctions are observed, particularly between Hindi, Dance, and Rap music.
Supervised Learning Approaches
We utilize NumPy, Pandas, XGBoost, and Scikit-Learn to explore different supervised techniques to classify the songs into their respective playlists.
We label encode the playlists, create train test splits, and scale each feature to prevent unfair influence. 
We model and classify audio features using Logistic Regression, achieving better than random results. 
We experiment with different learning rate parameters of Gradient Boosted Trees to reduce overfitting and increase test accuracy.
An overview of model performances
Final Supervised Model Design
We utilize PyTorch to create a 7-layered neural network trained with cross entropy loss, ReLU activation, and a 0.3 dropout rate.
We convert our data to torch tensors, create batches for efficient data loading, and compute class-weights for fair model evaluation. 
Network structure 
We use Cross Entropy Loss with an Adam optimizer. 
The model is trained on each train batch, with loss being calculated as the average losses across batches. 
Early stopping is implemented to prevent overfitting.
Unsupervised Approaches
We use Gaussian Mixture Models to explore whether the corpus can be clustered differently than how it was categorized in its supervised counterpart.
We note that clusters are not as distinct as expected, with songs of different playlists between clubbed together into the same clusters.
Conclusions
Given the high F1-score of our supervised model, it suffices to say that it has achieved our objectives of being able to classify songs into specific genres based on specific DJ preferences. If deployed, this model will save substantial amounts of time analyzing and categorizing songs.
It is important to note that the implementation of our model was limited by certain factors, such as the lack of data available and large computing resources. We hope to address these issues in our next steps.
Next Steps – Questions to Answer
- Which additional features are causing unsupervised techniques to cluster songs differently than their supervised counterparts?
- Can this algorithm be built up to decide which songs to download in the first place?
- How can we develop an algorithm to not just classify music, but also recommend which tracks to mix?