# Pastebin f2SbLPRB [2020-07-24 14:36:56,186] INFO in request_consumer: Request consumer started! [2020-07-24 14:37:25,455] INFO in request_consumer: Received a request! [2020-07-24 14:37:25,781] INFO in download: Downloading msid-mbid-mapping-with-matchable-20200630-525068.tar.bz2 from FTP... [2020-07-24 14:45:37,779] INFO in __init__: Verifying dump integrity... [2020-07-24 14:45:49,628] ERROR in request_consumer: Error in the query handler for query 'import.mapping': Received SHA256 checksum doesn't match the calculated checksum, aborting. Traceback (most recent call last): File "/rec/listenbrainz_spark/request_consumer/request_consumer.py", line 57, in get_result return query_handler(**params) File "/rec/listenbrainz_spark/request_consumer/jobs/import_dump.py", line 42, in import_mapping_to_hdfs src, mapping_name = ListenbrainzDataDownloader().download_msid_mbid_mapping(directory=temp_dir) File "/rec/listenbrainz_spark/ftp/download.py", line 130, in download_msid_mbid_mapping dest_path = self.download_dump(mapping_file_name, directory) File "/rec/listenbrainz_spark/ftp/__init__.py", line 87, in download_dump raise DumpInvalidException("Received SHA256 checksum doesn't match the calculated checksum, aborting.") listenbrainz_spark.exceptions.DumpInvalidException: Received SHA256 checksum doesn't match the calculated checksum, aborting. [2020-07-24 14:45:49,764] INFO in request_consumer: Request done! [2020-07-24 14:45:49,764] INFO in request_consumer: Received a request! [2020-07-24 14:46:31,970] INFO in create_dataframes: Preparing users data and saving to HDFS... [2020-07-24 14:46:55,193] INFO in create_dataframes: Preparing recordings data and saving to HDFS... [2020-07-24 14:47:56,872] INFO in create_dataframes: Preparing listen data dump and playcounts, saving playcounts to HDFS... [2020-07-24 14:50:16,333] DEBUG in request_consumer: Pushing result to RabbitMQ... [2020-07-24 14:50:16,334] INFO in request_consumer: Done! [2020-07-24 14:50:16,334] INFO in request_consumer: Number of messages sent: 1 [2020-07-24 14:50:16,334] INFO in request_consumer: Average size of message: 160 bytes [2020-07-24 14:50:16,334] INFO in request_consumer: Request done! [2020-07-24 14:50:16,335] INFO in request_consumer: Received a request! [2020-07-24 14:50:16,680] INFO in train_models: Splitting dataframe... [2020-07-24 14:50:24,678] INFO in train_models: Training models... [2020-07-24 14:51:29,801] INFO in train_models: Saving model... [2020-07-24 14:51:31,810] INFO in train_models: Model saved! [2020-07-24 14:51:31,829] INFO in train_models: Saving model metadata... [2020-07-24 14:51:32,143] INFO in train_models: Model metadata saved... [2020-07-24 14:51:32,160] INFO in train_models: Saving HTML... [2020-07-24 14:51:32,174] INFO in train_models: Done! [2020-07-24 14:51:32,174] DEBUG in request_consumer: Pushing result to RabbitMQ... [2020-07-24 14:51:32,175] INFO in request_consumer: Done! [2020-07-24 14:51:32,175] INFO in request_consumer: Number of messages sent: 1 [2020-07-24 14:51:32,175] INFO in request_consumer: Average size of message: 103 bytes [2020-07-24 14:51:32,175] INFO in request_consumer: Request done! [2020-07-24 14:51:32,175] INFO in request_consumer: Received a request! [2020-07-24 14:51:33,086] INFO in candidate_sets: Fetching listens to get top artists... [2020-07-24 14:51:33,121] INFO in candidate_sets: Fetching top artists... [2020-07-24 14:51:33,190] INFO in candidate_sets: Preparing top artists candidate set... [2020-07-24 14:51:33,224] INFO in candidate_sets: Fetching similar artists... [2020-07-24 14:51:33,283] INFO in candidate_sets: Preparing similar artists candidate set... [2020-07-24 14:51:33,317] INFO in candidate_sets: Saving candidate sets... [2020-07-24 14:51:48,891] INFO in candidate_sets: Done! [2020-07-24 14:52:00,253] INFO in candidate_sets: Saving HTML... [2020-07-24 14:52:00,443] INFO in candidate_sets: Done! [2020-07-24 14:52:00,452] DEBUG in request_consumer: Pushing result to RabbitMQ... [2020-07-24 14:52:00,453] INFO in request_consumer: Done! [2020-07-24 14:52:00,453] INFO in request_consumer: Number of messages sent: 1 [2020-07-24 14:52:00,453] INFO in request_consumer: Average size of message: 191 bytes [2020-07-24 14:52:00,453] INFO in request_consumer: Request done! [2020-07-24 14:52:00,453] INFO in request_consumer: Received a request! [2020-07-24 14:52:00,727] INFO in recommend: Loading model... [2020-07-24 14:52:03,360] INFO in recommend: Generating recommendations... [2020-07-24 14:52:22,162] INFO in recommend: Recommendations Generated! [2020-07-24 14:52:22,167] DEBUG in request_consumer: Pushing result to RabbitMQ... [2020-07-24 14:52:22,169] INFO in request_consumer: Done! [2020-07-24 14:52:22,170] INFO in request_consumer: Number of messages sent: 3 [2020-07-24 14:52:22,170] INFO in request_consumer: Average size of message: 16081 bytes [2020-07-24 14:52:22,170] INFO in request_consumer: Request done! [2020-07-24 14:52:36,763] INFO in request_consumer: Received a request!