music21.metadata.caching

JobProcessor

class music21.metadata.caching.JobProcessor

Processes metadata-caching jobs, either serially (e.g. single-threaded) or in parallel, as a generator.

Yields a dictionary of:

• MetadataEntry instances

• failed file paths

• the last processed file path

• the number of remaining jobs

>>> jobs = []
>>> mdb = corpus.corpora.CoreCorpus().search('monteverdi')[:3]
>>> paths = [x.sourcePath for x in mdb]

>>> for corpusPath in paths:
...     job = metadata.caching.MetadataCachingJob(
...         corpusPath,
...         parseUsingCorpus=True,
...         corpusName='core',
...         )
...     jobs.append(job)
>>> jobGenerator = metadata.caching.JobProcessor.process_serial(jobs)
>>> for result in jobGenerator:
...     print(result['remainingJobs'])
...
2
1
0

JobProcessor methods

static JobProcessor.process_parallel(jobs, processCount=None)

Process jobs in parallel, with processCount processes.

If processCount is none, use 1 fewer process than the number of available cores.

jobs is a list of MetadataCachingJob objects.

static JobProcessor.process_serial(jobs)

Process jobs serially.

MetadataCachingJob

class music21.metadata.caching.MetadataCachingJob(filePath, jobNumber=0, parseUsingCorpus=True, corpusName=None)

Parses one corpus path, and attempts to extract metadata from it:

>>> job = metadata.caching.MetadataCachingJob(
...     'bach/bwv66.6',
...     parseUsingCorpus=True,
...     corpusName='core',
...     )
>>> job.jobNumber
0
>>> job.corpusName
'core'
>>> job.run()
((<music21.metadata.bundles.MetadataEntry 'bach_bwv66_6'>,), ())
>>> results = job.getResults()
>>> errors = job.getErrors()

TODO: error list, not just numbers needs to be reported back up.

MetadataCachingJob read-only properties

MetadataCachingJob.cleanFilePath

MetadataCachingJob methods

MetadataCachingJob.getErrors()

MetadataCachingJob.getResults()

MetadataCachingJob.parseFilePath()

MetadataCachingJob.parseNonOpus(parsedObject)

MetadataCachingJob.parseOpus(parsedObject)

MetadataCachingJob.parseScoreInsideOpus(score, scoreNumber)

MetadataCachingJob.run()

WorkerProcess

class music21.metadata.caching.WorkerProcess(job_queue, result_queue)

A worker process for use by the multithreaded metadata-caching job processor.

WorkerProcess methods

WorkerProcess.run()

Method to be run in sub-process; can be overridden in sub-class

Functions

music21.metadata.caching.cacheMetadata(corpusNames=None, useMultiprocessing=True, verbose=False)

Cache metadata from corpora in corpusNames as local cache files.

Call as metadata.cacheMetadata()