message: Made small changes to the documentation.

diff:
@@ -19,19 +19,19 @@ class NMF_MFCC(mask_separation_base.MaskSeparationBase):
Non Negative Matrix Factorization using K-Means Clustering on MFCC (NMF MFCC) is a source separation
algorithm that runs Transformer NMF on the magnitude spectrogram of an input audio signal.
It uses K means clustering to cluster the templates and activations returned by the NMF. The dot product of the
- clustered templates and activations results in a magnitude spectrogram only containing the separated source.
- This is used to create a Binary Mask object, which can then be applied to return a list of Audio Signal objects
- corresponding to each separated source.
+ clustered templates and activations results in a magnitude spectrogram only containing a separated source.
+ This is used to create a Binary Mask object, and the whole process can be applied for each cluster to
+ return a list of Audio Signal objects corresponding to each separated source.
References:
Mel Frequency Cepstral Coefficients(MFCC): https://en.wikipedia.org/wiki/Mel-frequency_cepstrum
Volker Gnann, Martin Spiertz. "Source-filter based clustering for monaural blind source separation"
October 2002: https://www.researchgate.net/profile/Martin_Spiertz/publication/228567105_Source-filter_based
_clustering_for_monaural_blind_source_separation/links/0c960526b56dae0c69000000.pdf
- scikit-learn's KMeans: http://scikit-learn.org/stable/modules/generated/sklearn.cluster.KMeans.html
+ scikit-learn KMeans: http://scikit-learn.org/stable/modules/generated/sklearn.cluster.KMeans.html
Parameters:
- input_audio_signal (np.array): a 2-row Numpy matrix containing samples of the two-channel mixture.
+ input_audio_signal (np.array): An Audio Signal object of the input audio signal.
num_sources (int): Number of sources to find.
num_templates (int): Number of template vectors to used in NMF.
distance_measure (str): The type of distance measure to use in NMF - euclidean or divergence.
@@ -105,7 +105,7 @@ class NMF_MFCC(mask_separation_base.MaskSeparationBase):
the same order for each channel.
Returns:
- self.masks (np.array): A list of binary mask objects that can be used to extract the sources
+ self.masks (np.array): A list of binary mask objects that can be used to extract the sources.
Example:
@@ -114,14 +114,14 @@ class NMF_MFCC(mask_separation_base.MaskSeparationBase):
signal = nussl.AudioSignal(path_to_input_file='input_name.wav')
- # Set up and run Repet
+ # Set up and run NMF MFCC
nmf_mfcc = nussl.NMF_MFCC(signal, num_sources=2) # Returns a binary mask by default
masks = nmf_mfcc.run()
# Get audio signals
sources = nmf_mfcc.make_audio_signals()
- # output the background
+ # Output the sources
for i, source in enumerate(sources):
output_file_name = str(i) + '.wav'
source.write_audio_to_file(output_file_name)


message: Revised documentation for tests and nmf_mfcc.

change_type: MODIFY

diff:
@@ -17,44 +17,92 @@ import masks
class NMF_MFCC(mask_separation_base.MaskSeparationBase):
"""
Non Negative Matrix Factorization using K-Means Clustering on MFCC (NMF MFCC) is a source separation
- algorithm that runs Transformer NMF on the magnitude spectrogram of an input audio signal.
- It uses K means clustering to cluster the templates and activations returned by the NMF. The dot product of the
- clustered templates and activations results in a magnitude spectrogram only containing a separated source.
- This is used to create a Binary Mask object, and the whole process can be applied for each cluster to
- return a list of Audio Signal objects corresponding to each separated source.
+ algorithm that runs Transformer NMF on the magnitude spectrogram of an input audio signal. The templates matrix
+ is then converted to mel-space to reduce the dimensionality. The K means clustering then clusters the converted
+ templates and activations matrices. The dot product of the clustered templates and activations results in a
+ magnitude spectrogram only containing a separated source. This is used to create a Binary Mask object, and the
+ whole process can be applied for each cluster to return a list of Audio Signal objects corresponding to each
+ separated source.
References:
- Mel Frequency Cepstral Coefficients(MFCC): https://en.wikipedia.org/wiki/Mel-frequency_cepstrum
- Volker Gnann, Martin Spiertz. "Source-filter based clustering for monaural blind source separation"
- October 2002: https://www.researchgate.net/profile/Martin_Spiertz/publication/228567105_Source-filter_based
- _clustering_for_monaural_blind_source_separation/links/0c960526b56dae0c69000000.pdf
- scikit-learn KMeans: http://scikit-learn.org/stable/modules/generated/sklearn.cluster.KMeans.html
+ * `Mel Frequency Cepstral Coefficients (MFCC) <https://en.wikipedia.org/wiki/Mel-frequency_cepstrum>`_
+
+ * `scikit-learn KMeans <http://scikit-learn.org/stable/modules/generated/sklearn.cluster.KMeans.html>`_
+
+ * Spiertz, Martin, and Volker Gnann. "Source-filter based clustering for monaural blind source separation."
+ Proceedings of the 12th International Conference on Digital Audio Effects. 2009.
+
+ See Also:
+ :ref:`The NMF MFCC Demo Example <nmf_mfcc_demo>`
Parameters:
- input_audio_signal (np.array): An Audio Signal object of the input audio signal.
+ input_audio_signal (:class:`audio_signal.AudioSignal`): The :class:`audio_signal.AudioSignal` object that
+ NMF MFCC will be run on. This makes a copy of ``input_audio_signal``
num_sources (int): Number of sources to find.
- num_templates (int): Number of template vectors to used in NMF.
+ num_templates (int): Number of template vectors to used in NMF. Defaults to 50.
+ num_iterations (int): The number of iterations to go through in NMF. Defaults to 50.
+ random_seed (int): The seed to use in the numpy random generator in NMF and KMeans. See code examples below
+ for how this is used. Default uses no seed.
distance_measure (str): The type of distance measure to use in NMF - euclidean or divergence.
- num_iterations (int): The number of iterations to go through in NMF.
- random_seed (int): The seed to use in the numpy random generator
- kmeans_kwargs (dict): The kwargs for KMeans parameters.
- convert_to_mono (bool): Given a stereo signal, convert to mono.
- mask_type (object): A soft or binary mask object used for the source separation.
- mfcc_range (int/list/tuple): The range of mfcc to use for clustering.
- n_mfcc (int): The max number of mfccs to use.
+ Defaults to euclidean.
+ kmeans_kwargs (dict): The kwargs for KMeans parameters. Can be initialized with a dictionary of keys
+ corresponding to parameters in KMeans. See below for an example. Default is none.
+ convert_to_mono (bool): Given a stereo signal, convert to mono. Default set to False.
+ mask_type (str): A soft or binary mask object used for the source separation.
+ Defaults to Binary.
+ mfcc_range (int,list,tuple): The range of mfcc for clustering. See code examples below. Defaults to 1:14.
+ n_mfcc (int): The max number of mfccs to use. Defaults to 20.
Attributes:
- input_audio_signal (object): An Audio Signal object of the input audio signal.
- clusterer (object): A KMeans object for clustering the templates and activations.
- signal_stft (np.matrix): The stft data for the input audio signal.
- labeled_templates (np.array): A Numpy array containing the labeled templates columns
+ input_audio_signal (:class:`audio_signal.AudioSignal`): The :class:`audio_signal.AudioSignal` object that
+ NMF MFCC will be run on. This makes a copy of ``input_audio_signal``
+ clusterer (:obj:`KMeans`): A scikit-learn KMeans object for clustering the templates and activations.
+ signal_stft (:obj:`np.ndarray`): The stft data for the input audio signal.
+ labeled_templates (:obj:`list`): A Numpy array containing the labeled templates columns
from the templates matrix for a particular source.
- sources (np.array): A Numpy array containing the list of Audio Signal objects for each source.
- masks (np.array): A Numpy array containing the lists of Binary Mask objects for each channel.
+ sources (:obj:`list`): A list containing the lists of Audio Signal objects for each source.
+ result_masks (:obj:`list`): A list containing the lists of Binary Mask objects for each channel.
+
+ Initializing Example:
+
+ .. code-block:: python
+ :linenos:
+ # Initialize input signal
+ signal = nussl.AudioSignal(path_to_input_file='input_name.wav')
+
+ # Default initialization
+ nmf_mfcc = nussl.NMF_MFCC(signal, num_sources=2)
+
+ # Random Seeding initialization
+ # Set NMF and KMeans seeds to 0 (no kmeans_kwarg initialized, so random_seed is used for random_state)
+ nmf_mfcc = nussl.NMF_MFCC(signal, num_sources=2, random_seed=0)
+
+ # Use individual seeding
+ nmf_mfcc = nussl.NMF_MFCC(signal, num_sources=2, random_seed=0, kmeans_kwargs = {random_state: 1})
+
+ # Only set random_seed and not random_state
+ nmf_mfcc = nussl.NMF_MFCC(signal, num_sources=2, random_seed=0, kmeans_kwargs = {random_state: None})
+
+ # MFCC range initialization
+ # Set up the max of the MFCC range by only using an int
+ nmf_mfcc = nussl.NMF_MFCC(signal, num_sources=2, num_templates=6, mfcc_range=5)
+
+ # Set up the MFCC range by using a list [min, max]
+ nmf_mfcc = nussl.NMF_MFCC(signal, num_sources=2, num_templates=6, mfcc_range=[3, 15])
+
+ # Set up the MFCC range by using a tuple (min, max)
+ nmf_mfcc = nussl.NMF_MFCC(signal, num_sources=2, num_templates=6, mfcc_range=(2, 14))
+
+ # KMeans initialization
+ # Initialize all KMeans arguments, see KMeans documentation for more detail.
+ kmeans_kwargs = { n_clusters: 8, init: 'k-means++', n_init: 10, max_iter: 300, tol: 1e-4,
+ precompute_distances: 'auto', verbose: 0, random_state: None, copy_x: True,
+ n_jobs: 1, algorithm: 'auto'}
+ nmf_mfcc = nussl.NMF_MFCC(signal, num_sources=2, kmeans_kwargs=kmeans_kwargs)
"""
- def __init__(self, input_audio_signal, num_sources, num_templates=50, distance_measure='euclidean',
- num_iterations=50, random_seed=None, kmeans_kwargs=None, convert_to_mono=False,
+ def __init__(self, input_audio_signal, num_sources, num_templates=50, num_iterations=50, random_seed=None,
+ distance_measure=transformer_nmf.TransformerNMF.EUCLIDEAN, kmeans_kwargs=None, convert_to_mono=False,
mask_type=mask_separation_base.MaskSeparationBase.BINARY_MASK, mfcc_range=(1, 14), n_mfcc=20):
super(NMF_MFCC, self).__init__(input_audio_signal=input_audio_signal, mask_type=mask_type)
@@ -64,19 +112,15 @@ class NMF_MFCC(mask_separation_base.MaskSeparationBase):
self.num_iterations = num_iterations
self.random_seed = random_seed
self.kmeans_kwargs = kmeans_kwargs
- self.convert_to_mono = convert_to_mono
self.n_mfcc = n_mfcc
- self.mask_type = mask_type
self.signal_stft = None
- self.input_audio_signal = input_audio_signal
self.labeled_templates = None
self.sources = []
- self.masks = []
# Convert the stereo signal to mono if indicated
- if self.convert_to_mono:
- self.input_audio_signal.to_mono(overwrite=True, remove_channels=False)
+ if convert_to_mono:
+ self.audio_signal.to_mono(overwrite=True, remove_channels=False)
# Set the MFCC range
if isinstance(mfcc_range, int) and mfcc_range < n_mfcc:
@@ -84,7 +128,7 @@ class NMF_MFCC(mask_separation_base.MaskSeparationBase):
elif isinstance(mfcc_range, (tuple, list)) and len(mfcc_range) == 2:
self.mfcc_start, self.mfcc_end = mfcc_range[0], mfcc_range[1]
else:
- raise ValueError('mfcc_range is not set correctly! Must be a tuple or list with min and max, or int (max)')
+ raise ValueError('mfcc_range is not set correctly! Must be a tuple or list with (min, max), or int (max)')
# If kmeans_kwargs does not include the 'random_state', use the random_seed instead. Else, use the value
# provided in the dictionary. If kmeans_kwargs is None, use the random_seed.
@@ -101,11 +145,12 @@ class NMF_MFCC(mask_separation_base.MaskSeparationBase):
def run(self):
""" This function calls TransformerNMF on the magnitude spectrogram of each channel in the input audio signal.
The templates and activation matrices returned are clustered using K-Means clustering. These clusters are used
- to create mask objects for each source. Note: The masks are not returned in a particular order, but they are in
- the same order for each channel.
+ to create mask objects for each source. Note: The masks in self.result_masks are not returned in a particular
+ order corresponding to the sources, but they are in the same order for each channel.
Returns:
- self.masks (np.array): A list of binary mask objects that can be used to extract the sources.
+ result_masks (:obj:`list`): A list of :obj:`MaskBase`-derived objects for each source.
+ (to get a list of :obj:`AudioSignal`-derived objects run :func:`make_audio_signals`)
Example:
@@ -126,21 +171,18 @@ class NMF_MFCC(mask_separation_base.MaskSeparationBase):
output_file_name = str(i) + '.wav'
source.write_audio_to_file(output_file_name)
"""
- self.masks = []
- self.input_audio_signal.stft_params = self.stft_params
- self.signal_stft = self.input_audio_signal.stft()
+ self.result_masks = []
+ self.audio_signal.stft_params = self.stft_params
- for i in range(self.input_audio_signal.num_channels):
- channel_stft = self.input_audio_signal.get_magnitude_spectrogram_channel(i)
+ for i in range(self.audio_signal.num_channels):
+ channel_stft = self.audio_signal.get_magnitude_spectrogram_channel(i)
# Set up NMF and run
nmf = transformer_nmf.TransformerNMF(input_matrix=channel_stft, num_components=self.num_templates,
- seed=self.random_seed)
+ seed=self.random_seed, should_do_epsilon=False,
+ max_num_iterations=self.num_iterations,
+ distance_measure=self.distance_measure)
- # Pass these to constructor (above)
- nmf.should_use_epsilon = False
- nmf.max_num_iterations = self.num_iterations
- nmf.distance_measure = self.distance_measure
channel_activation_matrix, channel_templates_matrix = nmf.transform()
# Cluster the templates matrix into Mel frequencies and retrieve labels
@@ -151,25 +193,24 @@ class NMF_MFCC(mask_separation_base.MaskSeparationBase):
# Extract sources from signal
channel_masks = self._extract_masks(channel_stft, channel_templates_matrix, channel_activation_matrix)
- self.masks.append(channel_masks)
+ self.result_masks.append(channel_masks)
- return self.masks
+ return self.result_masks
- def _extract_masks(self, signal_stft, templates_matrix, activation_matrix):
+ def _extract_masks(self, templates_matrix, activation_matrix):
""" Creates binary masks from clustered templates and activation matrices
Parameters:
- signal_stft (np.matrix): A 2D Numpy matrix containing the stft of the current channel
- templates_matrix (np.matrix): A 2D Numpy matrix containing the templates matrix after running NMF on
+ templates_matrix (np.ndarray): A 2D Numpy array containing the templates matrix after running NMF on
the current channel
- activation_matrix (np.matrix): A 2D Numpy matrix containing the activation matrix after running NMF on
+ activation_matrix (np.ndarray): A 2D Numpy array containing the activation matrix after running NMF on
the current channel
Returns:
- channel_mask_list (np.matrix): A list of Binary Mask objects corresponding to each source
+ channel_mask_list (list): A list of Binary Mask objects corresponding to each source
"""
- if signal_stft is None:
+ if self.audio_signal.stft_data is None:
raise ValueError('Cannot extract masks with no signal_stft data')
self.sources = []
@@ -188,6 +229,7 @@ class NMF_MFCC(mask_separation_base.MaskSeparationBase):
music_stft_max = np.maximum(mask_matrix, np.abs(signal_stft))
mask_matrix = np.divide(mask_matrix, music_stft_max)
mask = np.nan_to_num(mask_matrix)
+
if self.mask_type == self.BINARY_MASK:
mask = np.round(mask)
mask_object = masks.BinaryMask(np.array(mask))
@@ -203,10 +245,10 @@ class NMF_MFCC(mask_separation_base.MaskSeparationBase):
self.sources (np.array): An array of audio_signal objects containing each separated source
"""
self.sources = []
- for i in range(self.input_audio_signal.num_channels):
- channel_mask = self.masks[i]
+ for i in range(self.audio_signal.num_channels):
+ channel_mask = self.result_masks[i]
for j in range(self.num_sources):
- channel_stft = self.input_audio_signal.get_stft_channel(i)
+ channel_stft = self.audio_signal.get_stft_channel(i)
source = self.audio_signal.make_copy_with_stft_data(channel_stft, verbose=False)
source = source.apply_mask(channel_mask[j])
source.stft_params = self.stft_params

diff:
@@ -10,6 +10,9 @@ import scipy.io
import os
import warnings
+
+# Freezes essential values from DUET in its current implementation for benchmarking
+# See test_benchmark_duet() in test_duet.py for usage
def freeze_duet_values():
signal = nussl.AudioSignal(os.path.join('..', 'Input', 'dev1_female3_inst_mix.wav'))
duet = nussl.Duet(signal, 3)

diff:
@@ -5,7 +5,9 @@ import numpy as np
import nussl
import os
-#Freeze key NMF MFCC values to the benchmark folder
+
+# Freezes essential values from NMF MFCC in its current implementation for benchmarking
+# See test_benchmark_nmf_mfcc() in test_nmf_mfcc.py for usage
def main():
path_to_benchmark_file = os.path.join('..', 'Input', 'piano_and_synth_arp_chord_mono.wav')
signal = nussl.AudioSignal(path_to_benchmark_file)

diff:
@@ -327,12 +327,12 @@ class AudioSignalUnitTests(unittest.TestCase):
signal = nussl.AudioSignal(path_to_input_file=input_file_name)
signal.stft_params = signal.stft_params
signal_stft = signal.stft()
- assert (signal_stft.shape[2] == 2)
+ assert (signal_stft.shape[nussl.constants.STFT_CHAN_INDEX] == 2)
signal.to_mono(overwrite=True)
signal.stft_params = signal.stft_params
signal_stft = signal.stft()
- assert (signal_stft.shape[2] == 1)
+ assert (signal_stft.shape[nussl.constants.STFT_CHAN_INDEX]== 1)
def test_stft(self):
"""

diff:
@@ -191,6 +191,13 @@ class DuetUnitTests(unittest.TestCase):
final_signals = duet.make_audio_signals()
+ # Is the audio data the same?
+ assert all(np.array_equal(benchmark_final_signals[i].audio_data, final_signals[i].audio_data)
+ for i in range(len(final_signals)))
+
+ # Check to see if AudioSignal's API changed; do we need to refreeze?
+ assert all(benchmark_final_signals[i] == final_signals[i] for i in range(len(final_signals)))
+
assert np.all(benchmark_final_signals == final_signals)
# test_smooth_matrix

diff:
@@ -21,12 +21,14 @@ class NMFMFCCUnitTests(unittest.TestCase):
benchmark_dict[key] = value
return benchmark_dict
- def test_mfcc_initialization(self):
+ def test_nmf_mfcc_initialization(self):
+ # Test initializing the MFCC range
+
# Load input file
input_file_name = os.path.join('..', 'input', 'piano_and_synth_arp_chord_mono.wav')
signal = nussl.AudioSignal(path_to_input_file=input_file_name)
- # Set up NMMF MFCC using an int
+ # Set up the max of the MFCC range by only using an int
nmf_mfcc = nussl.NMF_MFCC(signal, num_sources=2, num_templates=6, mfcc_range=5, num_iterations=5)
assert nmf_mfcc.mfcc_start == 1
assert nmf_mfcc.mfcc_end == 5
@@ -34,12 +36,12 @@ class NMFMFCCUnitTests(unittest.TestCase):
nmf_mfcc.run()
sources = nmf_mfcc.make_audio_signals()
- # Using a list
+ # Set up the MFCC range by using a list [min, max]
nmf_mfcc = nussl.NMF_MFCC(signal, num_sources=2, num_templates=6, mfcc_range=[3, 15], num_iterations=5)
assert nmf_mfcc.mfcc_start == 3
assert nmf_mfcc.mfcc_end == 15
- # Using a tuple
+ # Set up the MFCC range by using a tuple (min, max)
nmf_mfcc = nussl.NMF_MFCC(signal, num_sources=2, num_templates=6, mfcc_range=(2, 14), num_iterations=5)
assert nmf_mfcc.mfcc_start == 2
assert nmf_mfcc.mfcc_end == 14
@@ -97,9 +99,6 @@ class NMFMFCCUnitTests(unittest.TestCase):
# and run
nmf_mfcc.run()
sources = nmf_mfcc.make_audio_signals()
- for i, source in enumerate(sources):
- output_file_name = str(i) + '.wav'
- source.write_audio_to_file(output_file_name)
def test_piano_and_synth_stereo(self):
# Load input file
@@ -111,9 +110,6 @@ class NMFMFCCUnitTests(unittest.TestCase):
# and run
nmf_mfcc.run()
sources = nmf_mfcc.make_audio_signals()
- for i, source in enumerate(sources):
- output_file_name = str(i) + '.wav'
- source.write_audio_to_file(output_file_name)
def test_piano_and_synth_stereo_to_mono(self):
# Load input file
@@ -126,9 +122,6 @@ class NMFMFCCUnitTests(unittest.TestCase):
# and run
nmf_mfcc.run()
sources = nmf_mfcc.make_audio_signals()
- for i, source in enumerate(sources):
- output_file_name = str(i) + '.wav'
- source.write_audio_to_file(output_file_name)
def benchmark_nmf_mfcc(self):
# Load input file
@@ -137,16 +130,19 @@ class NMFMFCCUnitTests(unittest.TestCase):
# Set up NMMF MFCC
nmf_mfcc = nussl.NMF_MFCC(signal, num_sources=2, num_templates=6, distance_measure="euclidean",
- num_iterations=150, random_seed=0)
+ num_iterations=100, random_seed=0)
# and run
nmf_mfcc.run()
- benchmark_labeled_templates = np.load(os.path.join('nmf_mfcc_reference', 'test', 'benchmark_labeled_templates.npy'))
- benchmark_masks = np.load(os.path.join('nmf_mfcc_reference', 'test', 'benchmark_masks.npy'))
+ benchmark_labeled_templates = np.load(os.path.join('nmf_mfcc_reference', 'nmf_mfcc_benchmarks',
+ 'benchmark_labeled_templates.npy'))
+ benchmark_masks = np.load(os.path.join('nmf_mfcc_reference', 'nmf_mfcc_benchmarks', 'benchmark_masks.npy'))
nmf_mfcc.make_audio_signals()
- benchmark_sources = np.load(os.path.join('nmf_mfcc_reference', 'test', 'benchmark_sources.npy'))
+ benchmark_sources = np.load(os.path.join('nmf_mfcc_reference', 'nmf_mfcc_benchmarks', 'benchmark_sources.npy'))
assert np.all(benchmark_labeled_templates == nmf_mfcc.labeled_templates)
assert np.all(benchmark_masks == nmf_mfcc.masks)
- assert np.all(benchmark_sources == nmf_mfcc.sources)
+ for i in range(nmf_mfcc.num_sources):
+ assert np.all(benchmark_sources[i].audio_data == nmf_mfcc.sources[i].audio_data)
+ assert all(benchmark_sources[i] == nmf_mfcc.sources[i] for i in range(len(nmf_mfcc.sources)))

license: MIT License

repo: nussl/nussl


message: Fixed tests and variable naming bugs.

diff:
@@ -173,9 +173,10 @@ class NMF_MFCC(mask_separation_base.MaskSeparationBase):
"""
self.result_masks = []
self.audio_signal.stft_params = self.stft_params
+ self.audio_signal.stft()
- for i in range(self.audio_signal.num_channels):
- channel_stft = self.audio_signal.get_magnitude_spectrogram_channel(i)
+ for ch in range(self.audio_signal.num_channels):
+ channel_stft = self.audio_signal.get_magnitude_spectrogram_channel(ch)
# Set up NMF and run
nmf = transformer_nmf.TransformerNMF(input_matrix=channel_stft, num_components=self.num_templates,
@@ -192,12 +193,12 @@ class NMF_MFCC(mask_separation_base.MaskSeparationBase):
self.labeled_templates = self.clusterer.labels_
# Extract sources from signal
- channel_masks = self._extract_masks(channel_stft, channel_templates_matrix, channel_activation_matrix)
+ channel_masks = self._extract_masks(channel_templates_matrix, channel_activation_matrix, ch)
self.result_masks.append(channel_masks)
return self.result_masks
- def _extract_masks(self, templates_matrix, activation_matrix):
+ def _extract_masks(self, templates_matrix, activation_matrix, ch):
""" Creates binary masks from clustered templates and activation matrices
Parameters:
@@ -226,7 +227,7 @@ class NMF_MFCC(mask_separation_base.MaskSeparationBase):
activation_mask[i, :] = 0 if i in source_indices else activation_matrix[i, :]
mask_matrix = templates_mask.dot(activation_mask)
- music_stft_max = np.maximum(mask_matrix, np.abs(signal_stft))
+ music_stft_max = np.maximum(mask_matrix, np.abs(self.audio_signal.get_stft_channel(ch)))
mask_matrix = np.divide(mask_matrix, music_stft_max)
mask = np.nan_to_num(mask_matrix)

change_type: ADD

change_type: RENAME

diff:
@@ -14,7 +14,7 @@ import warnings
# Freezes essential values from DUET in its current implementation for benchmarking
# See test_benchmark_duet() in test_duet.py for usage
def freeze_duet_values():
- signal = nussl.AudioSignal(os.path.join('..', 'Input', 'dev1_female3_inst_mix.wav'))
+ signal = nussl.AudioSignal(os.path.join('..', 'input', 'dev1_female3_inst_mix.wav'))
duet = nussl.Duet(signal, 3)
output_folder = os.path.abspath('duet_reference/duet_benchmarks')

diff:
@@ -9,19 +9,21 @@ import os
# Freezes essential values from NMF MFCC in its current implementation for benchmarking
# See test_benchmark_nmf_mfcc() in test_nmf_mfcc.py for usage
def main():
- path_to_benchmark_file = os.path.join('..', 'Input', 'piano_and_synth_arp_chord_mono.wav')
+ path_to_benchmark_file = os.path.join('..', '..', 'input', 'piano_and_synth_arp_chord_mono.wav')
signal = nussl.AudioSignal(path_to_benchmark_file)
# Set random seed in NMF and KMeans to 0
nmf_mfcc = nussl.NMF_MFCC(signal, num_sources=2, num_templates=6, distance_measure="euclidean",
- num_iterations=100, random_seed=0)
- output_folder = os.path.abspath('nmf_mfcc_reference/new')
+ num_iterations=10, random_seed=0)
+ output_folder = os.path.abspath('new')
nmf_mfcc.run()
np.save(os.path.join(output_folder, "benchmark_labeled_templates"), nmf_mfcc.labeled_templates)
- np.save(os.path.join(output_folder, "benchmark_masks"), nmf_mfcc.masks)
+ np.save(os.path.join(output_folder, "benchmark_masks"), nmf_mfcc.result_masks)
nmf_mfcc.make_audio_signals()
+ for source in nmf_mfcc.sources:
+ source.path_to_input_file = ''
np.save(os.path.join(output_folder, "benchmark_sources"), nmf_mfcc.sources)
if __name__ == '__main__':

diff:
+9e4a4d58a4b8299c26c7525cc53cb0cdcbb26525
\ No newline at end of file

diff:
+dc8a536e7eb325f0fec4a7495b63b02c498debb5
\ No newline at end of file

diff:
@@ -9,7 +9,7 @@ import os
class NMFMFCCUnitTests(unittest.TestCase):
# Update this if the benchmark file changes and rerun freeze_duet_values() (below)
- path_to_benchmark_file = os.path.join('..', 'Input', 'piano_and_synth_arp_chord_mono.wav')
+ path_to_benchmark_file = os.path.join('..', 'input', 'piano_and_synth_arp_chord_mono.wav')
def load_benchmarks(self):
benchmark_dict = {}
@@ -85,20 +85,19 @@ class NMFMFCCUnitTests(unittest.TestCase):
nmf_mfcc.run()
sources = nmf_mfcc.make_audio_signals()
- for i, source in enumerate(sources):
- output_file_name = str(i) + '.wav'
- source.write_audio_to_file(output_file_name)
-
def test_piano_and_synth(self):
# Load input file
input_file_name = os.path.join('..', 'input', 'piano_and_synth_arp_chord_mono.wav')
signal = nussl.AudioSignal(path_to_input_file=input_file_name)
# Set up NMMF MFCC
- nmf_mfcc = nussl.NMF_MFCC(signal, num_sources=2, num_templates=6, num_iterations=50, random_seed=0)
+ nmf_mfcc = nussl.NMF_MFCC(signal, num_sources=2, num_templates=6, num_iterations=10, random_seed=0)
# and run
nmf_mfcc.run()
sources = nmf_mfcc.make_audio_signals()
+ for i, source in enumerate(sources):
+ output_file_name = str(i) + '.wav'
+ source.write_audio_to_file(output_file_name)
def test_piano_and_synth_stereo(self):
# Load input file
@@ -130,19 +129,19 @@ class NMFMFCCUnitTests(unittest.TestCase):
# Set up NMMF MFCC
nmf_mfcc = nussl.NMF_MFCC(signal, num_sources=2, num_templates=6, distance_measure="euclidean",
- num_iterations=100, random_seed=0)
+ num_iterations=10, random_seed=0)
# and run
nmf_mfcc.run()
- benchmark_labeled_templates = np.load(os.path.join('nmf_mfcc_reference', 'nmf_mfcc_benchmarks',
+ benchmark_labeled_templates = np.load(os.path.join('nmf_mfcc_reference', 'new',
'benchmark_labeled_templates.npy'))
- benchmark_masks = np.load(os.path.join('nmf_mfcc_reference', 'nmf_mfcc_benchmarks', 'benchmark_masks.npy'))
+ benchmark_masks = np.load(os.path.join('nmf_mfcc_reference', 'new', 'benchmark_masks.npy'))
nmf_mfcc.make_audio_signals()
- benchmark_sources = np.load(os.path.join('nmf_mfcc_reference', 'nmf_mfcc_benchmarks', 'benchmark_sources.npy'))
+ for source in nmf_mfcc.sources:
+ source.path_to_input_file = ''
+ benchmark_sources = np.load(os.path.join('nmf_mfcc_reference', 'new', 'benchmark_sources.npy'))
+ assert np.all(benchmark_sources == nmf_mfcc.sources)
assert np.all(benchmark_labeled_templates == nmf_mfcc.labeled_templates)
- assert np.all(benchmark_masks == nmf_mfcc.masks)
- for i in range(nmf_mfcc.num_sources):
- assert np.all(benchmark_sources[i].audio_data == nmf_mfcc.sources[i].audio_data)
- assert all(benchmark_sources[i] == nmf_mfcc.sources[i] for i in range(len(nmf_mfcc.sources)))
+ assert np.all(benchmark_masks == nmf_mfcc.result_masks)

original_message: Fixed tests and variable naming bugs.


message: refactored functions

diff:
@@ -199,8 +199,8 @@ class AudioSignal(object):
self.stft_params = stft_params
# Effects
- self._ffmpeg_effects = []
- self._sox_effects = []
+ self._ffmpeg_effects_chain = []
+ self._sox_effects_chain = []
def __str__(self):
dur = f'{self.signal_duration:0.3f}' if self.signal_duration else '[unknown]'
@@ -1653,79 +1653,86 @@ class AudioSignal(object):
# Effect Hooks #
##################################
- def build_effect(self, reset=True):
- if not self._ffmpeg_effects and not self._sox_effects:
+ def build_effect(self, reset=True, overwrite=False):
+ if not self._ffmpeg_effects_chain and not self._sox_effects_chain:
raise RuntimeError("No effect hooks have been called on this AudioSignal")
new_signal = self
- if self._ffmpeg_effects:
- new_signal = effects.build_effects_ffmpeg(new_signal, self._ffmpeg_effects)
- if self._sox_effects:
- new_signal = effects.build_effects_sox(new_signal, self._sox_effects)
+ if self._ffmpeg_effects_chain:
+ new_signal = effects.build_effects_ffmpeg(new_signal, self._ffmpeg_effects_chain)
+ if self._sox_effects_chain:
+ new_signal = effects.build_effects_sox(new_signal, self._sox_effects_chain)
if reset:
- self._ffmpeg_effects = []
- self._sox_effects = []
+ self.reset_effects_chain()
+ if overwrite:
+ self.audio_data = new_signal.audio_data
+ return self
+
return new_signal
+ def reset_effects_chain(self):
+ self._ffmpeg_effects_chain = []
+ self._sox_effects_chain = []
+
def time_stretch(self, factor):
- self._sox_effects.append(effects.time_stretch(factor))
+ self._sox_effects_chain.append(effects.time_stretch(factor))
return self
def pitch_shift(self, shift):
- self._sox_effects.append(effects.pitch_shift(shift))
+ self._sox_effects_chain.append(effects.pitch_shift(shift))
return self
def low_pass(self, freq, poles=2, width_type="h", width=.707):
- self._ffmpeg_effects.append(effects.low_pass(freq, poles=poles,
+ self._ffmpeg_effects_chain.append(effects.low_pass(freq, poles=poles,
width_type=width_type, width=width))
return self
def high_pass(self, freq, poles=2, width_type="h", width=.707):
- self._ffmpeg_effects.append(effects.high_pass(freq, poles=poles,
+ self._ffmpeg_effects_chain.append(effects.high_pass(freq, poles=poles,
width_type=width_type, width=width))
return self
def tremolo(self, mod_freq, mod_depth):
- self._ffmpeg_effects.append(effects.tremolo(mod_freq, mod_depth))
+ self._ffmpeg_effects_chain.append(effects.tremolo(mod_freq, mod_depth))
return self
def vibrato(self, mod_freq, mod_depth):
- self._ffmpeg_effects.append(effects.vibrato(mod_freq, mod_depth))
+ self._ffmpeg_effects_chain.append(effects.vibrato(mod_freq, mod_depth))
return self
def chorus(self, delays, decays, speeds, depths, in_gain=.4, out_gain=.4):
- self._ffmpeg_effects.append(effects.chorus(delays, decays,
+ self._ffmpeg_effects_chain.append(effects.chorus(delays, decays,
speeds, depths, in_gain=.4, out_gain=.4))
return self
def phaser(self, in_gain=.4, out_gain=.74, delay=3, decay=.4,
speed=.5, _type="triangular"):
- self._ffmpeg_effects.append(effects.phaser(in_gain=.4, out_gain=.74, delay=3,
+ self._ffmpeg_effects_chain.append(effects.phaser(in_gain=.4, out_gain=.74, delay=3,
decay=.4, speed=.5, _type="triangular"))
return self
def flanger(self, delay=0, depth=2, regen=0, width=71,
speed=.5, phase=25, shape="sinusoidal", interp="linear"):
- self._ffmpeg_effects.append(effects.flanger(delay=0, depth=2, regen=0, width=71,
+ self._ffmpeg_effects_chain.append(effects.flanger(delay=0, depth=2, regen=0, width=71,
speed=.5, phase=25, shape="sinusoidal", interp="linear"))
return self
def emphasis(self, level_in, level_out, _type="col", mode='production'):
- self._ffmpeg_effects.append(effects.emphasis(level_in,
+ self._ffmpeg_effects_chain.append(effects.emphasis(level_in,
level_out, _type="col", mode='production'))
return self
def compressor(self, level_in, mode="downward", reduction_ratio=2,
attack=20, release=250, makeup=1, knee=2.8284, link="average",
detection="rms", mix=1, threshold=.125):
- self._ffmpeg_effects.append(effects.compressor(level_in, mode="downward", reduction_ratio=2,
+ self._ffmpeg_effects_chain.append(effects.compressor(level_in, mode="downward", reduction_ratio=2,
attack=20, release=250, makeup=1, knee=2.8284, link="average",
detection="rms", mix=1, threshold=.125))
return self
def equalizer(self, bands):
- self._ffmpeg_effects.append(effects.equalizer(bands))
+ self._ffmpeg_effects_chain.append(effects.equalizer(bands))
return self

diff:
@@ -11,11 +11,37 @@ from pysndfx import AudioEffectsChain
from .constants import LEVEL_MIN, LEVEL_MAX
from .utils import _close_temp_files
-def _create_ffmpeg_filter(_filter, **kwargs):
- filter_function = lambda ffmpeg_stream: ffmpeg_stream.filter(_filter, **kwargs)
- return filter_function
+class FilterFunction():
+ def __init__(self, _filter, **filter_kwargs):
+ self.func = None
+ raise NotImplementedError
+
+ def __call__(self, stream):
+ return self.func(stream)
+
+class FFmpegFilter(FilterFunction):
+ def __init__(self, _filter, **filter_kwargs):
+ self.filter = _filter
+ self.func = lambda stream: stream.filter(_filter, **filter_kwargs)
+
+class SoXFilter(FilterFunction):
+ def __init__(self, _filter, **filter_kwargs):
+ self.filter = _filter
+ if _filter == "tempo":
+ self.func = lambda stream: stream.tempo(**filter_kwargs)
+ elif _filter == "pitch":
+ self.func = lambda stream: stream.pitch(**filter_kwargs)
+ else:
+ raise ValueError
def build_effects_ffmpeg(audio_signal, filters, silent=False):
+ """
+ Applies list of ffmpeg_filters to AudioSignal, then creates a copy of the AudioSignal,
+ with said filters applied
+ Args:
+ audio_signal: AudioSignal object
+ filters: List of filters, returned by
+ """
# lazy load
from .audio_signal import AudioSignal
@@ -66,19 +92,7 @@ def time_stretch(factor):
if not np.issubdtype(type(factor), np.number) or factor <= 0:
raise ValueError("stretch_factor must be a positve scalar")
- # stretched_audio_data = []
-
- # for channel in audio_signal.get_channels():
- # stretched_audio_data.append(librosa.effects.time_stretch(channel, stretch_factor))
- # stretched_audio_data = np.array(stretched_audio_data)
-
- # # The following line causes a UserWarning.
- # stretched_signal = audio_signal.make_copy_with_audio_data(stretched_audio_data)
-
- # # Alternative line to suppress UserWarning
- # # stretched_signal = AudioSignal(audio_data_array=stretched_audio_data)
-
- return lambda sox_stream: sox_stream.tempo(factor)
+ return SoXFilter("tempo", factor=factor)
def pitch_shift(shift):
"""
@@ -93,15 +107,7 @@ def pitch_shift(shift):
if not isinstance(shift, int):
raise ValueError("shift must be an integer.")
- # sample_rate = audio_signal.sample_rate
- # shifted_audio_data = []
-
- # for channel in audio_signal.get_channels():
- # shifted_audio_data.append(librosa.effects.pitch_shift(channel, sample_rate, shift))
- # shifted_audio_data = np.array(shifted_audio_data)
- # shifted_signal = audio_signal.make_copy_with_audio_data(shifted_audio_data)
-
- return lambda sox_stream: sox_stream.pitch(shift)
+ return SoXFilter("pitch", shift=shift)
def low_pass(freq, poles=2, width_type="h", width=.707):
"""
@@ -125,7 +131,7 @@ def low_pass(freq, poles=2, width_type="h", width=.707):
# raise ValueError("mix must be between 0 and 1")
- return _create_ffmpeg_filter("lowpass", f=freq, p=poles,
+ return FFmpegFilter("lowpass", f=freq, p=poles,
t=width_type, w=width)
def high_pass(freq, poles=2, width_type="h", width=.707):
@@ -151,7 +157,7 @@ def high_pass(freq, poles=2, width_type="h", width=.707):
# raise ValueError("mix must be between 0 and 1")
- return _create_ffmpeg_filter("highpass", f=freq, p=poles,
+ return FFmpegFilter("highpass", f=freq, p=poles,
t=width_type, w=width)
def tremolo(mod_freq, mod_depth):
@@ -172,7 +178,7 @@ def tremolo(mod_freq, mod_depth):
if not np.issubdtype(type(mod_depth), np.number) or mod_depth < 0 or mod_depth > 1:
raise ValueError("mod_depth should be positve scalar between 0 and 1.")
- return _create_ffmpeg_filter("tremolo", f=mod_freq, d=mod_depth)
+ return FFmpegFilter("tremolo", f=mod_freq, d=mod_depth)
def vibrato(mod_freq, mod_depth):
"""
@@ -192,7 +198,7 @@ def vibrato(mod_freq, mod_depth):
if not np.issubdtype(type(mod_depth), np.number) or mod_depth < 0 or mod_depth > 1:
raise ValueError("mod_depth should be positve scalar between 0 and 1.")
- return _create_ffmpeg_filter("vibrato", f=mod_freq, d=mod_depth)
+ return FFmpegFilter("vibrato", f=mod_freq, d=mod_depth)
def chorus(delays, decays, speeds, depths,
in_gain=.4, out_gain=.4):
@@ -230,7 +236,7 @@ def chorus(delays, decays, speeds, depths,
decays = make_arglist_ffmpeg(decays)
depths = make_arglist_ffmpeg(depths)
- return _create_ffmpeg_filter("chorus", in_gain=in_gain,
+ return FFmpegFilter("chorus", in_gain=in_gain,
out_gain=out_gain, delays=delays, speeds=speeds, decays=decays, depths=depths)
def phaser(in_gain=.4, out_gain=.74, delay=3,
@@ -261,7 +267,7 @@ def phaser(in_gain=.4, out_gain=.74, delay=3,
"type": _type
}
- return _create_ffmpeg_filter("aphaser", in_gain = in_gain,
+ return FFmpegFilter("aphaser", in_gain = in_gain,
out_gain = out_gain, delay = delay, speed = speed, decay = decay, **type_kwarg)
@@ -316,7 +322,7 @@ def flanger(delay=0, depth=2, regen=0, width=71,
_flanger_argcheck(delay, depth, regen, width, speed, phase, shape, interp)
- return _create_ffmpeg_filter("flanger", delay=delay,
+ return FFmpegFilter("flanger", delay=delay,
depth=depth, regen=regen, width=width, speed=speed, phase=phase, shape=shape,
interp=interp)
@@ -352,7 +358,7 @@ def emphasis(level_in, level_out, _type="col", mode='production'):
'type': _type
}
- return _create_ffmpeg_filter("aemphasis", level_in=level_in, level_out=level_out, mode=mode, **type_kwarg)
+ return FFmpegFilter("aemphasis", level_in=level_in, level_out=level_out, mode=mode, **type_kwarg)
def _compressor_argcheck(level_in, mode, reduction_ratio,
attack, release, makeup, knee, link,
@@ -428,7 +434,7 @@ def compressor(level_in, mode="downward", reduction_ratio=2,
attack, release, makeup, knee, link, detection, mix, threshold)
# TODO: for some reason the mode arg doesn't work in ffmpeg. figure out why
- return _create_ffmpeg_filter("acompressor", level_in=level_in,
+ return FFmpegFilter("acompressor", level_in=level_in,
ratio=reduction_ratio, attack=attack, release=release, makeup=makeup,
knee=knee, link=link, detection=detection, mix=mix, threshold=threshold)
@@ -471,4 +477,4 @@ def equalizer(bands):
for band in bands
])
- return _create_ffmpeg_filter("anequalizer", params=params)
+ return FFmpegFilter("anequalizer", params=params)

original_message: refactored functions


message: documentation in progress

diff:
@@ -1657,6 +1657,7 @@ class AudioSignal(object):
if not self._ffmpeg_effects_chain and not self._sox_effects_chain:
raise RuntimeError("No effect hooks have been called on this AudioSignal")
+ # TODO: SOX before ffmpeg
new_signal = self
if self._ffmpeg_effects_chain:
new_signal = effects.build_effects_ffmpeg(new_signal, self._ffmpeg_effects_chain)

diff:
@@ -12,6 +12,10 @@ from .constants import LEVEL_MIN, LEVEL_MAX
from .utils import _close_temp_files
class FilterFunction():
+ """
+ The FilterFunction class is an abstract class for functions that take
+ audio processing streams, such as ffmpeg-python and pysndfx
+ """
def __init__(self, _filter, **filter_kwargs):
self.func = None
raise NotImplementedError
@@ -36,11 +40,16 @@ class SoXFilter(FilterFunction):
def build_effects_ffmpeg(audio_signal, filters, silent=False):
"""
- Applies list of ffmpeg_filters to AudioSignal, then creates a copy of the AudioSignal,
- with said filters applied
+ build_effects_ffmpeg takes an AudioSignal object and a list of FFmpegFilter objects
+ and sequentially applies each filter to the signal.
Args:
audio_signal: AudioSignal object
- filters: List of filters, returned by
+ filters: List of FFmpegFilter objects
+ silent: If True, suppresses all FFmpeg output. If False, FFmpeg will log information
+ with loglevel 'info'
+ Returns:
+ augmented_signal: A new AudioSignal object, with the audio data from
+ audio_signal after applying filters
"""
# lazy load
@@ -68,6 +77,16 @@ def build_effects_ffmpeg(audio_signal, filters, silent=False):
return augmented_signal
def build_effects_sox(audio_signal, filters):
+ """
+ build_effects_sox takes an AudioSignal object and a list of SoXFilter objects
+ and sequentially applies each filter to the signal.
+ Args:
+ audio_signal: AudioSignal object
+ filters: List of SoXFilter objects
+ Returns:
+ augmented_signal: A new AudioSignal object, with the audio data from
+ audio_signal after applying filters
+ """
audio_data = audio_signal.audio_data
chain = AudioEffectsChain()
@@ -82,12 +101,12 @@ def make_arglist_ffmpeg(lst, sep="|"):
def time_stretch(factor):
"""
- Linear Stretch on the time axis
+ Returns a SoXFilter, when called on an pysndfx stream, will multiply the
+ tempo of the audio by factor.
Args:
- audio_signal: An AudioSignal object
- factor_range: A tuple of length 2. Denotes start and end of possible ranges for factor.
+ factor: Scaling factor for tempo change. Must be positive.
Returns:
- stretched_signal: A copy of the original audio signal, with augmentations applied
+ filter: A SoXFilter object, to be called on an pysndfx stream
"""
if not np.issubdtype(type(factor), np.number) or factor <= 0:
raise ValueError("stretch_factor must be a positve scalar")
@@ -96,13 +115,13 @@ def time_stretch(factor):
def pitch_shift(shift):
"""
- Pitch shift on the frequency axis
+ Returns a SoXFilter, when called on an pysndfx stream, will increase the pitch
+ of the audio by a number of cents, denoted in shift.
Args:
- audio_signal: An AudioSignal object
shift: The number of cents (1/100th of a half step) to shift the audio.
Positive values increases the frequency of the signal
Returns:
- shifted_signal: A copy of the original audio signal, with augmentations applied
+ filter: A SoXFilter object, to be called on an pysndfx stream
"""
if not isinstance(shift, int):
raise ValueError("shift must be an integer.")
@@ -111,12 +130,21 @@ def pitch_shift(shift):
def low_pass(freq, poles=2, width_type="h", width=.707):
"""
- Creates high pass filter.
+ https://ffmpeg.org/ffmpeg-all.html#lowpass
+
+ Creates an FFmpegFilter object, which when called on an ffmpeg stream,
+ applies a low pass filter to the audio signal.
Args:
- audio_signal: An AudioSignal object
- highest_freq: Threshold for high pass. Should be positive scalar
+ freq: Threshold for high pass. Should be positive scalar
+ poles: Number of poles. should be either 1 or 2
+ width_type: Unit of width for filter. Must be either:
+ 'h': Hz
+ 'q': Q-factor
+ 'o': octave
+ 's': slope
+ 'k': kHz
Returns:
- augmented_signal: A copy of the original audio signal, with augmentations applied
+ filter: A FFmpegFilter object, to be called on an ffmpeg stream
"""
if not np.issubdtype(type(freq), np.number) or freq <= 0:
raise ValueError("lowest_freq should be positive scalar")
@@ -136,12 +164,21 @@ def low_pass(freq, poles=2, width_type="h", width=.707):
def high_pass(freq, poles=2, width_type="h", width=.707):
"""
- Creates high pass filter.
+ https://ffmpeg.org/ffmpeg-all.html#highpass
+
+ Creates a FFmpegFilter object, when called on an ffmpeg stream,
+ applies a high pass filter to the audio signal.
Args:
- audio_signal: An AudioSignal object
- highest_freq: Threshold for high pass. Should be positive scalar
+ freq: Threshold for high pass. Should be positive scalar
+ poles: Number of poles. should be either 1 or 2
+ width_type: Unit of width for filter. Must be either:
+ 'h': Hz
+ 'q': Q-factor
+ 'o': octave
+ 's': slope
+ 'k': kHz
Returns:
- augmented_signal: A copy of the original audio signal, with augmentations applied
+ filter: A FFmpegFilter object, to be called on an ffmpeg stream
"""
if not np.issubdtype(type(freq), np.number) or freq <= 0:
raise ValueError("lowest_freq should be positive scalar")
@@ -157,20 +194,19 @@ def high_pass(freq, poles=2, width_type="h", width=.707):
# raise ValueError("mix must be between 0 and 1")
- return FFmpegFilter("highpass", f=freq, p=poles,
- t=width_type, w=width)
+ return FFmpegFilter("highpass", f=freq, p=poles, t=width_type, w=width)
def tremolo(mod_freq, mod_depth):
"""
https://ffmpeg.org/ffmpeg-all.html#tremolo
- Applies tremolo filter
+ Creates a FFmpegFilter object, when called on an ffmpeg stream,
+ applies a tremolo filter to the audio signal
Args:
- audio_signal: An AudioSignal object
mod_freq: Modulation frequency. Must be between .1 and 20000.
mod_depth: Modulation depth. Must be between 0 and 1.
Returns:
- augmented_signal: A copy of the original audio signal, with augmentations applied
+ filter: A FFmpegFilter object, to be called on an ffmpeg stream
"""
if not np.issubdtype(type(mod_freq), np.number) or mod_freq < .1 or mod_freq > 20000:
raise ValueError("mod_freq should be positive scalar between .1 and 20000")
@@ -184,13 +220,13 @@ def vibrato(mod_freq, mod_depth):
"""
https://ffmpeg.org/ffmpeg-all.html#vibrato
- Applies vibrato filter
+ Creates a FFmpegFilter object, when called on an ffmpeg stream,
+ applies a vibrato filter to the audio signal
Args:
- audio_signal: An AudioSignal object
mod_freq: Modulation frequency. Must be between .1 and 20000.
mod_depth: Modulation depth. Must be between 0 and 1.
Returns:
- augmented_signal: A copy of the original audio signal, with augmentations applied
+ filter: A FFmpegFilter object, to be called on an ffmpeg stream
"""
if not np.issubdtype(type(mod_freq), np.number) or mod_freq < .1 or mod_freq > 20000:
raise ValueError("mod_freq should be positve scalar between .1 and 20000")
@@ -205,18 +241,17 @@ def chorus(delays, decays, speeds, depths,
"""
https://ffmpeg.org/ffmpeg-all.html#chorus
- Applies Chorus Filter(s). Delays, decays, speeds, and depths
- must all lists of the same length.
+ Creates a FFmpegFilter object, when called on an ffmpeg stream,
+ applies a vibrato filter to the audio signal
Args:
- audio_signal: An AudioSignal object
input_gain: Proportion of input gain
output_gain: Proportion of output gain
delays: list of delays in ms. Typical Delay is 40ms-6ms
- decays: list of decays
- speeds: list of speeds
- depths: list of depths
+ decays: list of decays. Must be between 0 and 1
+ speeds: list of speeds. Must be between 0 and 1
+ depths: list of depths. Must be between 0 and 1
Returns:
- augmented_signal: A copy of the original audio signal, with augmentations applied
+ filter: A FFmpegFilter object, to be called on an ffmpeg stream
"""
if (in_gain > 1 or in_gain < 0 or out_gain > 1 or out_gain < 0):
raise ValueError("in_gain and out_gain must be between 0 and 1.")
@@ -230,6 +265,7 @@ def chorus(delays, decays, speeds, depths,
!= len(speeds) or len(speeds) != len(depths)):
raise ValueError("Delays, decays, depths, and speeds must all be the same length.")
+ # TODO: Add more argchecks
delays = make_arglist_ffmpeg(delays)
speeds = make_arglist_ffmpeg(speeds)

diff:
@@ -345,18 +345,18 @@ def test_hooks(mix_and_sources, check_against_regression_data):
}])
)
- assert len(signal._ffmpeg_effects) == 10
- assert len(signal._sox_effects) == 2
+ assert len(signal._ffmpeg_effects_chain) == 10
+ assert len(signal._sox_effects_chain) == 2
augmented_signal = signal.build_effect(reset=False)
- assert len(signal._ffmpeg_effects) == 10
- assert len(signal._sox_effects) == 2
+ assert len(signal._ffmpeg_effects_chain) == 10
+ assert len(signal._sox_effects_chain) == 2
augmented_signal = signal.build_effect()
- assert len(signal._ffmpeg_effects) == 0
- assert len(signal._sox_effects) == 0
+ assert len(signal._ffmpeg_effects_chain) == 0
+ assert len(signal._sox_effects_chain) == 0
with pytest.raises(RuntimeError):
signal.build_effect()

original_message: documentation in progress


message: 100 percent coverage

diff:
@@ -1654,20 +1654,23 @@ class AudioSignal(object):
##################################
def build_effect(self, reset=True, overwrite=False):
+ """
+ Builds all effects
+ """
if not self._ffmpeg_effects_chain and not self._sox_effects_chain:
raise RuntimeError("No effect hooks have been called on this AudioSignal")
- # TODO: SOX before ffmpeg
new_signal = self
- if self._ffmpeg_effects_chain:
- new_signal = effects.build_effects_ffmpeg(new_signal, self._ffmpeg_effects_chain)
if self._sox_effects_chain:
new_signal = effects.build_effects_sox(new_signal, self._sox_effects_chain)
+ if self._ffmpeg_effects_chain:
+ new_signal = effects.build_effects_ffmpeg(new_signal, self._ffmpeg_effects_chain)
if reset:
self.reset_effects_chain()
if overwrite:
self.audio_data = new_signal.audio_data
+ self.stft_data = None
return self
return new_signal

diff:
@@ -17,7 +17,6 @@ class FilterFunction():
audio processing streams, such as ffmpeg-python and pysndfx
"""
def __init__(self, _filter, **filter_kwargs):
- self.func = None
raise NotImplementedError
def __call__(self, stream):

diff:
@@ -316,6 +316,9 @@ def test_misc_param_check():
with pytest.raises(ValueError):
effects.equalizer([band])
+ with pytest.raises(ValueError):
+ effects.SoXFilter('fail')
+
def test_silent_mode(mix_and_sources):
signal, _ = mix_and_sources
effects.build_effects_ffmpeg(signal, [], silent=True)
@@ -353,7 +356,12 @@ def test_hooks(mix_and_sources, check_against_regression_data):
assert len(signal._ffmpeg_effects_chain) == 10
assert len(signal._sox_effects_chain) == 2
- augmented_signal = signal.build_effect()
+ augmented_signal = signal.build_effect(reset=False, overwrite=True)
+
+ reg_path = path.join(REGRESSION_PATH, "hooks.json")
+ fx_regression(augmented_signal.audio_data, reg_path, check_against_regression_data)
+
+ augmented_signal = augmented_signal.build_effect()
assert len(signal._ffmpeg_effects_chain) == 0
assert len(signal._sox_effects_chain) == 0
@@ -361,6 +369,3 @@ def test_hooks(mix_and_sources, check_against_regression_data):
with pytest.raises(RuntimeError):
signal.build_effect()
- reg_path = path.join(REGRESSION_PATH, "hooks.json")
-
- fx_regression(augmented_signal.audio_data, reg_path, check_against_regression_data)

change_type: DELETE

diff: Binary files a/tests/utils/plots/viz_sources_dict.png and /dev/null differ

diff: Binary files a/tests/utils/plots/viz_sources_list.png and /dev/null differ

diff: Binary files a/tests/utils/plots/viz_spectrogram.png and /dev/null differ

diff: Binary files a/tests/utils/plots/viz_waveform.png and /dev/null differ

original_message: 100 percent coverage


message: Getting Errno. 22 on line 112 of test_effects.py

diff:
@@ -7,6 +7,8 @@ import warnings
import ffmpeg
import tempfile
from pysndfx import AudioEffectsChain
+import av
+from av.filter import Filter, Graph
from .constants import LEVEL_MIN, LEVEL_MAX
from .utils import _close_temp_files
@@ -23,9 +25,17 @@ class FilterFunction():
return self.func(stream)
class FFmpegFilter(FilterFunction):
+ @staticmethod
+ def _dict_to_ffmpeg_kwargs(dictionary):
+ str_params = []
+ for key, value in dictionary.items():
+ str_params.append(f"{key}={value}")
+ return ":".join(str_params)
+
def __init__(self, _filter, **filter_kwargs):
self.filter = _filter
- self.func = lambda stream: stream.filter(_filter, **filter_kwargs)
+ self.func = lambda graph: graph.add(_filter,
+ FFmpegFilter._dict_to_ffmpeg_kwargs(filter_kwargs))
class SoXFilter(FilterFunction):
def __init__(self, _filter, **filter_kwargs):
@@ -56,23 +66,65 @@ def build_effects_ffmpeg(audio_signal, filters, silent=False):
tmpfiles = []
- input_args = {}
- if silent:
- input_args['loglevel'] = 'quiet'
+ # input_args = {}
+ # if silent:
+ # input_args['loglevel'] = 'quiet'
with _close_temp_files(tmpfiles):
curr_tempfile = tempfile.NamedTemporaryFile(suffix=".wav")
- audio_signal.write_audio_to_file(curr_tempfile)
+ output_tempfile = tempfile.NamedTemporaryFile(suffix=".wav")
tmpfiles.append(curr_tempfile)
- stream = ffmpeg.input(curr_tempfile.name, **input_args)
- for _filter in filters:
- stream = _filter(stream)
- (stream
- .output(curr_tempfile.name)
- .overwrite_output()
- .run()
+ tmpfiles.append(output_tempfile)
+ audio_signal.write_audio_to_file(curr_tempfile)
+
+ in_cont = av.open(curr_tempfile.name, mode='r')
+ out_cont = av.open(output_tempfile.name, mode='w')
+
+ in_stream = in_cont.streams.audio[0]
+ out_stream = out_cont.add_stream(
+ codec_name=in_stream.codec_context.codec,
+ rate=audio_signal.sample_rate
+ )
+ # out_stream = out_count.add_stream(template=in_stream)
+
+ buffer_args = "sample_rate=%d:sample_fmt=%s:channel_layout=%s:time_base=%d/%d" % (
+ in_stream.rate, in_stream.format.name,
+ in_stream.layout.name,
+ 1, in_stream.rate,
)
- augmented_signal = AudioSignal(path_to_input_file=curr_tempfile.name)
+
+ print(buffer_args)
+
+ graph = Graph()
+ fchain = [graph.add('abuffer', buffer_args)]
+ for _filter in filters:
+ fchain.append(_filter(graph))
+ fchain[-2].link_to(fchain[-1])
+
+ fchain.append(graph.add("abuffersink"))
+ fchain[-2].link_to(fchain[-1])
+
+ graph.configure()
+
+ for in_frame in in_cont.decode(in_stream):
+ graph.push(in_frame)
+ out_frame = graph.pull()
+ for out_packet in out_stream.encode(out_frame):
+ out_cont.mux(out_packet)
+
+ out_cont.close()
+
+ # tmpfiles.append(curr_tempfile)
+ # stream = ffmpeg.input(curr_tempfile.name, **input_args)
+ # for _filter in filters:
+ # stream = _filter(stream)
+ # (stream
+ # .output(curr_tempfile.name)
+ # .overwrite_output()
+ # .run()
+ # )
+
+ augmented_signal = AudioSignal(path_to_input_file=output_tempfile.name)
return augmented_signal
def build_effects_sox(audio_signal, filters):

diff:
@@ -8,6 +8,7 @@ import os
import os.path as path
import librosa
import pytest
+import logging
REGRESSION_PATH="tests/core/regression/augmentation"
os.makedirs(REGRESSION_PATH, exist_ok=True)
@@ -53,6 +54,7 @@ def test_params(mix_and_sources):
effects.pitch_shift("this is a string")
def test_tremolo(mix_and_sources, check_against_regression_data):
+ logging.basicConfig()
f = 15
d = .5
signal, _ = mix_and_sources


message: reverted to ffmpeg-python

diff:
@@ -7,8 +7,8 @@ import warnings
import ffmpeg
import tempfile
from pysndfx import AudioEffectsChain
-import av
-from av.filter import Filter, Graph
+# import av
+# from av.filter import Filter, Graph
from .constants import LEVEL_MIN, LEVEL_MAX
from .utils import _close_temp_files
@@ -25,17 +25,20 @@ class FilterFunction():
return self.func(stream)
class FFmpegFilter(FilterFunction):
- @staticmethod
- def _dict_to_ffmpeg_kwargs(dictionary):
- str_params = []
- for key, value in dictionary.items():
- str_params.append(f"{key}={value}")
- return ":".join(str_params)
+ # @staticmethod
+ # def _dict_to_ffmpeg_kwargs(dictionary):
+ # str_params = []
+ # for key, value in dictionary.items():
+ # str_params.append(f"{key}={value}")
+ # return ":".join(str_params)
def __init__(self, _filter, **filter_kwargs):
self.filter = _filter
- self.func = lambda graph: graph.add(_filter,
- FFmpegFilter._dict_to_ffmpeg_kwargs(filter_kwargs))
+ # This code is for using PyAV.
+ # I was not able to make it work :(
+ # self.func = lambda graph: graph.add(_filter,
+ # FFmpegFilter._dict_to_ffmpeg_kwargs(filter_kwargs))
+ self.func = lambda stream: stream.filter(_filter, **filter_kwargs)
class SoXFilter(FilterFunction):
def __init__(self, _filter, **filter_kwargs):
@@ -66,65 +69,66 @@ def build_effects_ffmpeg(audio_signal, filters, silent=False):
tmpfiles = []
- # input_args = {}
- # if silent:
- # input_args['loglevel'] = 'quiet'
+ input_args = {}
+ if silent:
+ input_args['loglevel'] = 'quiet'
with _close_temp_files(tmpfiles):
curr_tempfile = tempfile.NamedTemporaryFile(suffix=".wav")
- output_tempfile = tempfile.NamedTemporaryFile(suffix=".wav")
+
tmpfiles.append(curr_tempfile)
- tmpfiles.append(output_tempfile)
audio_signal.write_audio_to_file(curr_tempfile)
- in_cont = av.open(curr_tempfile.name, mode='r')
- out_cont = av.open(output_tempfile.name, mode='w')
+ # This is for applying effects via PyAV
+ # I was not able to make it work :(
+ # Leaving the code for someone else to try to fix
- in_stream = in_cont.streams.audio[0]
- out_stream = out_cont.add_stream(
- codec_name=in_stream.codec_context.codec,
- rate=audio_signal.sample_rate
- )
- # out_stream = out_count.add_stream(template=in_stream)
+ # output_tempfile = tempfile.NamedTemporaryFile(suffix=".wav")
+ # tmpfiles.append(output_tempfile)
- buffer_args = "sample_rate=%d:sample_fmt=%s:channel_layout=%s:time_base=%d/%d" % (
- in_stream.rate, in_stream.format.name,
- in_stream.layout.name,
- 1, in_stream.rate,
- )
+ # in_cont = av.open(curr_tempfile.name, mode='r')
+ # out_cont = av.open(output_tempfile.name, mode='w')
- print(buffer_args)
+ # in_stream = in_cont.streams.audio[0]
+ # out_stream = out_cont.add_stream(
+ # codec_name=in_stream.codec_context.codec,
+ # rate=audio_signal.sample_rate
+ # )
- graph = Graph()
- fchain = [graph.add('abuffer', buffer_args)]
- for _filter in filters:
- fchain.append(_filter(graph))
- fchain[-2].link_to(fchain[-1])
+ # graph = Graph()
+ # fchain = [graph.add_abuffer(sample_rate=in_stream.rate,
+ # format=in_stream.format.name, layout=in_stream.layout.name,
+ # channels=audio_signal.num_channels)]
+ # for _filter in filters:
+ # fchain.append(_filter(graph))
+ # fchain[-2].link_to(fchain[-1])
- fchain.append(graph.add("abuffersink"))
- fchain[-2].link_to(fchain[-1])
+ # fchain.append(graph.add("abuffersink"))
+ # fchain[-2].link_to(fchain[-1])
- graph.configure()
+ # graph.configure()
- for in_frame in in_cont.decode(in_stream):
- graph.push(in_frame)
- out_frame = graph.pull()
- for out_packet in out_stream.encode(out_frame):
- out_cont.mux(out_packet)
+ # for ifr in in_cont.decode(in_stream):
+ # graph.push(ifr)
+ # ofr = graph.pull()
+ # ofr.pts = None
+ # for p in out_stream.encode(ofr):
+ # out_cont.mux(p)
- out_cont.close()
+ # out_cont.close()
+ # augmented_signal = AudioSignal(path_to_input_file=output_tempfile.name)
- # tmpfiles.append(curr_tempfile)
- # stream = ffmpeg.input(curr_tempfile.name, **input_args)
- # for _filter in filters:
- # stream = _filter(stream)
- # (stream
- # .output(curr_tempfile.name)
- # .overwrite_output()
- # .run()
- # )
+ tmpfiles.append(curr_tempfile)
+ stream = ffmpeg.input(curr_tempfile.name, **input_args)
+ for _filter in filters:
+ stream = _filter(stream)
+ (stream
+ .output(curr_tempfile.name)
+ .overwrite_output()
+ .run()
+ )
- augmented_signal = AudioSignal(path_to_input_file=output_tempfile.name)
+ augmented_signal = AudioSignal(path_to_input_file=curr_tempfile.name)
return augmented_signal
def build_effects_sox(audio_signal, filters):

original_message: reverted to ffmpeg-python


message: deleted augmentation.py

diff:
-import numpy as np
-import numpy.random as random
-import librosa
-import os
-from .audio_signal import AudioSignal
-import tempfile
-from .constants import LEVEL_MIN, LEVEL_MAX
-import warnings
-from .utils import _close_temp_files
-import ffmpeg
-import tempfile
-
-def apply_ffmpeg_filter(audio_signal, _filter, silent=True, **kwargs):
- tmpfiles = []
- with _close_temp_files(tmpfiles):
- input_tempfile = tempfile.NamedTemporaryFile(suffix=".wav")
- output_tempfile = tempfile.NamedTemporaryFile(suffix=".wav")
- tmpfiles.append(input_tempfile)
- tmpfiles.append(output_tempfile)
-
- audio_signal.write_audio_to_file(input_tempfile)
-
- if silent:
- input_kwargs = {'loglevel': 'quiet'}
- else:
- input_kwargs = {}
-
- (
- ffmpeg
- .input(input_tempfile.name, **input_kwargs)
- .filter(_filter, **kwargs)
- .output(output_tempfile.name)
- .overwrite_output()
- .run()
- )
-
- augmented_signal = AudioSignal(path_to_input_file=output_tempfile.name)
- output_tempfile.close()
- input_tempfile.close()
- return augmented_signal
-
-def make_arglist_ffmpeg(lst):
- return "|".join([str(s) for s in lst])
-
-def time_stretch(audio_signal, stretch_factor):
- """
- Linear Stretch on the time axis
- Args:
- audio_signal: An AudioSignal object
- factor_range: A tuple of length 2. Denotes start and end of possible ranges for factor.
- Returns:
- stretched_signal: A copy of the original audio signal, with augmentations applied
- """
- if not np.issubdtype(type(stretch_factor), np.number) or stretch_factor <= 0:
- raise ValueError("stretch_factor must be a positve scalar")
-
- stretched_audio_data = []
-
- for audio_row in audio_signal.get_channels():
- stretched_audio_data.append(librosa.effects.time_stretch(audio_row, stretch_factor))
- stretched_audio_data = np.array(stretched_audio_data)
-
- # The following line causes a UserWarning.
- stretched_signal = audio_signal.make_copy_with_audio_data(stretched_audio_data)
-
- # Alternative line to suppress UserWarning
- # stretched_signal = AudioSignal(audio_data_array=stretched_audio_data)
-
- return stretched_signal
-
-def pitch_shift(audio_signal, shift):
- """
- Pitch shift on the frequency axis
- Args:
- audio_signal: An AudioSignal object
- shift: The number of half-steps to shift the audio.
- Positive values increases the frequency of the signal
- Returns:
- shifted_signal: A copy of the original audio signal, with augmentations applied
- """
- if not isinstance(shift, int):
- raise ValueError("shift must be an integer.")
-
- sample_rate = audio_signal.sample_rate
- shifted_audio_data = []
-
- for audio_row in audio_signal.get_channels():
- shifted_audio_data.append(librosa.effects.pitch_shift(audio_row, sample_rate, shift))
- shifted_audio_data = np.array(shifted_audio_data)
- shifted_signal = audio_signal.make_copy_with_audio_data(shifted_audio_data)
-
- return shifted_signal
-
-def low_pass(audio_signal, highest_freq):
- """
- Applies low pass filter.
- Args:
- audio_signal: An AudioSignal object
- highest_freq: Threshold for low pass. Should be positive scalar
- Returns:
- augmented_signal: A copy of the original audio signal, with augmentations applied
- """
- if not np.issubdtype(type(highest_freq), np.number) or highest_freq <= 0:
- raise ValueError("highest_freq should be positve scalar")
-
- if audio_signal.stft_data is None:
- audio_signal.stft()
- l_stft = audio_signal.stft_data.copy()
- freq_vector = audio_signal.freq_vector
- idx = (np.abs(freq_vector - highest_freq)).argmin()
- l_stft[idx:, :, :] = 0
- augmented_signal = audio_signal.make_copy_with_stft_data(l_stft)
- augmented_signal.istft()
-
- return augmented_signal
-
-def high_pass(audio_signal, lowest_freq):
- """
- Applies high pass filter
- Args:
- audio_signal: An AudioSignal object
- highest_freq: Threshold for high pass. Should be positive scalar
- Returns:
- augmented_signal: A copy of the original audio signal, with augmentations applied
- """
- if not np.issubdtype(type(lowest_freq), np.number) or lowest_freq <= 0:
- raise ValueError("lowest_freq should be positive scalar")
-
- if audio_signal.stft_data is None:
- audio_signal.stft()
- h_stft = audio_signal.stft_data.copy()
- freq_vector = audio_signal.freq_vector
- idx = (np.abs(freq_vector - lowest_freq)).argmin()
- h_stft[:idx, :, :] = 0
- augmented_signal = audio_signal.make_copy_with_stft_data(h_stft)
- augmented_signal.istft()
-
- return augmented_signal
-
-def tremolo(audio_signal, mod_freq, mod_depth):
- """
- https://ffmpeg.org/ffmpeg-all.html#tremolo
-
- Applies tremolo filter
- Args:
- audio_signal: An AudioSignal object
- mod_freq: Modulation frequency. Must be between 0 and 1.
- mod_depth: Modulation depth. Must be between 0 and 1.
- Returns:
- augmented_signal: A copy of the original audio signal, with augmentations applied
- """
- if not np.issubdtype(type(mod_freq), np.number) or mod_freq < 0:
- raise ValueError("mod_freq should be positive scalar")
-
- if not np.issubdtype(type(mod_depth), np.number) or mod_depth < 0 or mod_depth > 1:
- raise ValueError("mod_depth should be positve scalar between 0 and 1.")
-
- return apply_ffmpeg_filter(audio_signal, "tremolo", f=mod_freq, d=mod_depth)
-
-def vibrato(audio_signal, mod_freq, mod_depth):
- """
- https://ffmpeg.org/ffmpeg-all.html#vibrato
-
- Applies vibrato filter
- Args:
- audio_signal: An AudioSignal object
- mod_freq: Modulation frequency. Must be between 0 and 1.
- mod_depth: Modulation depth. Must be between 0 and 1.
- Returns:
- augmented_signal: A copy of the original audio signal, with augmentations applied
- """
- if not np.issubdtype(type(mod_freq), np.number) or mod_freq < 0:
- raise ValueError("mod_freq should be positve scalar")
-
- if not np.issubdtype(type(mod_depth), np.number) or mod_depth < 0 or mod_depth > 1:
- raise ValueError("mod_depth should be positve scalar between 0 and 1.")
-
- return apply_ffmpeg_filter(audio_signal, "vibrato", f=mod_freq, d=mod_depth)
-
-def chorus(audio_signal, delays, decays, speeds, depths,
- in_gain=.4, out_gain=.4):
- """
- https://ffmpeg.org/ffmpeg-all.html#chorus
-
- Applies Chorus Filter(s). Delays, decays, speeds, and depths
- must all lists of the same length.
- Args:
- audio_signal: An AudioSignal object
- input_gain: Proportion of input gain
- output_gain: Proportion of output gain
- delays: list of delays in ms. Typical Delay is 40ms-6ms
- decays: list of decays
- speeds: list of speeds
- depths: list of depths
- Returns:
- augmented_signal: A copy of the original audio signal, with augmentations applied
- """
- if (in_gain > 1 or in_gain < 0 or out_gain > 1 or out_gain < 0):
- raise ValueError("in_gain and out_gain must be between 0 and 1.")
-
- # Bounds could not be found in ffmpeg docs, but recommendations were given
- if min(delays) < 0 or max(delays) > 1000:
- warnings.warn("One or more delays is far from the "
- "typical 40-60 ms range. This might produce strange results.", UserWarning)
-
- if (len(delays) != len(decays) or len(decays)
- != len(speeds) or len(speeds) != len(depths)):
- raise ValueError("Delays, decays, depths, and speeds must all be the same length.")
-
-
- delays = make_arglist_ffmpeg(delays)
- speeds = make_arglist_ffmpeg(speeds)
- decays = make_arglist_ffmpeg(decays)
- depths = make_arglist_ffmpeg(depths)
-
- return apply_ffmpeg_filter(audio_signal, "chorus", in_gain=in_gain,
- out_gain=out_gain, delays=delays, speeds=speeds, decays=decays, depths=depths)
-
-def phaser(audio_signal, in_gain=.4, out_gain=.74, delay=3,
- decay=.4, speed=.5, _type="triangular"):
- """
- https://ffmpeg.org/ffmpeg-all.html#aphaser
-
- Applies Phaser Filter.
- Args:
- audio_signal: An AudioSignal object
- input_gain: Proportion of input gain
- output_gain: Proportion of output gain
- delay: Delay of chorus filter. (Time between original signal and delayed)
- speed: Speed of the delayed filter.
- _type: modulation type
- Returns:
- augmented_signal: A copy of the original audio signal, with augmentations applied
- """
- if (in_gain > 1 or in_gain < 0 or out_gain > 1 or out_gain < 0):
- raise ValueError("in_gain and out_gain must be between 0 and 1.")
-
- allowed_mod_types = {"triangular", "sinusoidal", "t", "s"}
- if _type not in allowed_mod_types:
- raise ValueError(f"_type must be one of the following:\n{allowed_mod_types}")
-
- # type is reserved word in python, kwarg dict is necessary
- type_kwarg = {
- "type": _type
- }
-
- return apply_ffmpeg_filter(audio_signal, "aphaser", in_gain = in_gain,
- out_gain = out_gain, delay = delay, speed = speed, decay = decay, **type_kwarg)
-
-
-def _flanger_argcheck(delay, depth, regen, width,
- speed, phase, shape, interp):
-
- allowed_shape_types = {"triangular", "sinusoidal"}
- if shape not in allowed_shape_types:
- raise ValueError(f"shape must be one of the following:\n{allowed_shape_types}")
- allowed_interp_types = {"linear", "quadratic"}
- if interp not in allowed_interp_types:
- raise ValueError(f"interp must be one of the following:\n{allowed_interp_types}")
- if (delay < 0 or delay > 30
- or depth < 0 or depth > 10
- or regen < -95 or regen > 95
- or width < 0 or width > 100
- or speed < .1 or speed > 10):
- raise ValueError("One of the follow values are not in the accepted ranges"
- f"delay: {delay}\n"
- f"depth: {depth}\n"
- f"regen: {regen}\n"
- f"width: {width}\n"
- f"speed: {speed}\n"
- "The following are the bounds for the parameters to flanger()"
- "0 < delay < 30\n"
- "0 < depth < 10\n"
- "-95 < regen < 95\n"
- "0 < width < 100\n"
- ".1 < speed < 10\n"
- )
-
-def flanger(audio_signal, delay=0, depth=2, regen=0, width=71,
- speed=.5, phase=25, shape="sinusoidal", interp="linear"):
- """
- https://ffmpeg.org/ffmpeg-all.html#flanger
-
- Args:
- audio_signal: An AudioSignal object
- delay: Base delay in ms between original signal and copy.
- Must be between 0 and 30.
- depth: Sweep delay in ms. Must be between 0 and 10.
- regen: Percentage regeneration, or delayed signal feedback.
- Must be between -95 and 95.
- width: Percentage of delayed signal. Must be between 0 and 100.
- speed: Sweeps per second. Must be in .1 to 10
- shape: Swept wave shape, can be triangular or sinusoidal.
- phase: swept wave percentage-shift for multi channel. Must be between 0 and 100.
- interp: Delay Line interpolation. Must be linear of quadratic
- Returns:
- augmented_signal: A copy of the original audio signal, with augmentations applied
- """
-
- _flanger_argcheck(delay, depth, regen, width, speed, phase, shape, interp)
-
- return apply_ffmpeg_filter(audio_signal, "flanger", delay=delay,
- depth=depth, regen=regen, width=width, speed=speed, phase=phase, shape=shape,
- interp=interp)
-
-def emphasis(audio_signal, level_in, level_out, _type="col", mode='production'):
- """
- https://ffmpeg.org/ffmpeg-all.html#aemphasis
-
- Applies the emphasis filter, which either creates or restores
- signals from various physical mediums.
- Args:
- audio_signal: An AudioSignal object
- level_in: Input gain
- level_out: Output gain
- _type: medium type to convert/deconvert from
- mode: reproduction to convert from physical medium,
- production to convert to physical medium.
-
- Returns:
- augmented_signal: A copy of the original audio signal, with augmentations applied
- """
- if level_in < LEVEL_MIN or level_in > LEVEL_MAX \
- or level_out < LEVEL_MIN or level_out > LEVEL_MAX:
- raise ValueError(f"level_in and level_out must both be between {LEVEL_MIN} AND {LEVEL_MAX}")
- allowed_types={"col", "emi", "bsi", "riaa", "cd",
- "50fm", "75fm", "50kf", "75kf"}
- if _type not in allowed_types:
- raise ValueError(f"Given emphasis filter type is not supported by ffmpeg")
- if mode != "production" and mode != "reproduction":
- raise ValueError(f"mode must be production or reproduction")
-
- # type is a reserved word in python, so kwarg dict is necessary
- type_kwarg = {
- 'type': _type
- }
-
- return apply_ffmpeg_filter(audio_signal, "aemphasis", level_in=level_in, level_out=level_out, mode=mode, **type_kwarg)
-
-def _compressor_argcheck(level_in, mode, reduction_ratio,
- attack, release, makeup, knee, link,
- detection, mix, threshold):
-
- # The following values are taken from ffmpeg documentation
- if (level_in < LEVEL_MIN or level_in > LEVEL_MAX or
- mode not in {"upward", "downward"} or
- reduction_ratio < 1 or reduction_ratio > 20 or
- attack < .01 or attack > 2000 or
- release < .01 or release > 9000 or
- makeup < 1 or makeup > 64 or
- knee < 1 or knee > 8 or
- link not in {"average", "maximum"} or
- detection not in {"peak", "rms"} or
- mix < 0 or mix > 1 or
- threshold < 9.7563e-5 or threshold > 1):
- raise ValueError("One or more of the values provided are not within the bounds of the acompressor function"
- f"mode: {mode}\n"
- f"reduction_ratio: {reduction_ratio}\n"
- f"attack: {attack}\n"
- f"release: {release}\n"
- f"makeup: {makeup}\n"
- f"knee: {knee}\n"
- f"link: {link}\n"
- f"detection: {detection}\n"
- f"mix: {mix}\n"
- "The following are the bounds for these parameters:\n"
- f"{LEVEL_MIN} < level_in < {LEVEL_MAX}\n"
- "mode must be in {'upward, 'downward'}\n"
- "1 < reduction_ratio < 20\n"
- ".01 < attack < 2000\n"
- ".01 < release < 9000\n"
- "1 < makeup < 64\n"
- "1 < knee < 8\n"
- "link must be in {'average', 'maximum'}\n"
- "detection must be in {'peak', 'rms'}\n"
- "0 < mix < 1\n"
- " .000097563 < threshold < 1\n"
- )
-
-def compressor(audio_signal, level_in, mode="downward", reduction_ratio=2,
- attack=20, release=250, makeup=1, knee=2.8284, link="average",
- detection="rms", mix=1, threshold=.125):
- """
- https://ffmpeg.org/ffmpeg-all.html#acompressor
-
- Applies the compressor filter to an audio signal.
- See ffmpeg documentation for bounds
- Args:
- audio_signal: An AudioSignal object
- level_in: Input Gain
- mode: Mode of compressor operation. Can either be "upward" or "downward".
- threshold: Volume threshold. If a signal's volume is above the threshold,
- gain reduction would apply.
- reduction_ratio: Ratio in which the signal is reduced.
- attack: Time in ms between when the signal rises above threshold and when
- reduction is applied
- release: Time in ms between when the signal fall below threshold and
- when reduction is decreased.
- makeup: Factor of amplification post-processing
- knee: Softens the transition between reduction and lack of thereof.
- Higher values translate to a softer transition.
- link: Choose average between all channels or mean. String of either
- "average" or "mean.
- detection: Whether to process exact signal of the RMS of nearby signals.
- Either "peak" for exact or "rms".
- mix: Proportion of compressed signal in output.
- Returns:
- augmented_signal: A copy of the original audio signal, with augmentations applied
- """
- _compressor_argcheck(level_in, mode, reduction_ratio,
- attack, release, makeup, knee, link, detection, mix, threshold)
-
- # TODO: for some reason the mode arg doesn't work in ffmpeg. figure out why
- return apply_ffmpeg_filter(audio_signal, "acompressor", level_in=level_in,
- ratio=reduction_ratio, attack=attack, release=release, makeup=makeup,
- knee=knee, link=link, detection=detection, mix=mix, threshold=threshold)
-
-
-def equalizer(audio_signal, bands):
- """
- https://ffmpeg.org/ffmpeg-all.html#anequalizer
-
- Applies equalizer filter(s) to the audio_signal.
- Args:
- audio_signal: An AudioSignal object
- bands: A list of dictionaries, for each band. The required values for each dictionary:
- 'chn': List of channel numbers to apply filter
- 'f': central freqency of band
- 'w': Width of the band in Hz
- 'g': Band gain in dB
- 't': Set filter type for band, optional, can be:
- 0, for Butterworth
- 1, for Chebyshev type 1
- 2, for Chebyshev type 2
- """
- for band in bands:
- if max(band["chn"]) > audio_signal.num_channels:
- raise ValueError("band[\"chn\"] contains a number greater than number of channels")
- if band["f"] <= 0:
- raise ValueError("band[\"f\"] must be a positive scalar")
- if band["w"] <= 0:
- raise ValueError("band[\"w\"] must be a positive scalar")
- if band["g"] <= 0:
- raise ValueError("band[\"g\"] must be a positive scalar")
- if band["t"] not in {0, 1, 2}:
- raise ValueError("band[\"t\"] must be in {0, 1, 2}")
-
- params = make_arglist_ffmpeg([
- make_arglist_ffmpeg([
- f"c{c} f={band['f']} w={band['w']} g={band['g']} t={band['t']}"
- for c in band["chn"]
- ])
- for band in bands
- ])
-
- return apply_ffmpeg_filter(audio_signal, "anequalizer", params)
-
-

original_message: deleted augmentation.py


message: cleaned up for pep8

diff:
import numpy as np
-import numpy.random as random
-from librosa import load
-import os
-import tempfile
import warnings
import ffmpeg
import tempfile
from pysndfx import AudioEffectsChain
+from abc import abstractmethod
# import av
# from av.filter import Filter, Graph
from .constants import LEVEL_MIN, LEVEL_MAX
from .utils import _close_temp_files
+
class FilterFunction():
"""
The FilterFunction class is an abstract class for functions that take
audio processing streams, such as ffmpeg-python and pysndfx
"""
- def __init__(self, _filter, **filter_kwargs):
- raise NotImplementedError
def __call__(self, stream):
return self.func(stream)
+ def func(self, stream):
+ pass
+
+
class FFmpegFilter(FilterFunction):
# @staticmethod
# def _dict_to_ffmpeg_kwargs(dictionary):
@@ -40,6 +40,7 @@ class FFmpegFilter(FilterFunction):
# FFmpegFilter._dict_to_ffmpeg_kwargs(filter_kwargs))
self.func = lambda stream: stream.filter(_filter, **filter_kwargs)
+
class SoXFilter(FilterFunction):
def __init__(self, _filter, **filter_kwargs):
self.filter = _filter
@@ -50,6 +51,7 @@ class SoXFilter(FilterFunction):
else:
raise ValueError
+
def build_effects_ffmpeg(audio_signal, filters, silent=False):
"""
build_effects_ffmpeg takes an AudioSignal object and a list of FFmpegFilter objects
@@ -131,6 +133,7 @@ def build_effects_ffmpeg(audio_signal, filters, silent=False):
augmented_signal = AudioSignal(path_to_input_file=curr_tempfile.name)
return augmented_signal
+
def build_effects_sox(audio_signal, filters):
"""
build_effects_sox takes an AudioSignal object and a list of SoXFilter objects
@@ -151,9 +154,11 @@ def build_effects_sox(audio_signal, filters):
return audio_signal.make_copy_with_audio_data(augmented_data)
+
def make_arglist_ffmpeg(lst, sep="|"):
return sep.join([str(s) for s in lst])
+
def time_stretch(factor):
"""
Returns a SoXFilter, when called on an pysndfx stream, will multiply the
@@ -168,6 +173,7 @@ def time_stretch(factor):
return SoXFilter("tempo", factor=factor)
+
def pitch_shift(shift):
"""
Returns a SoXFilter, when called on an pysndfx stream, will increase the pitch
@@ -183,6 +189,20 @@ def pitch_shift(shift):
return SoXFilter("pitch", shift=shift)
+def _pass_arg_check(freq, poles, width_type, width):
+ if not np.issubdtype(type(freq), np.number) or freq <= 0:
+ raise ValueError("lowest_freq should be positive scalar")
+ if poles not in {1, 2}:
+ raise ValueError("poles must be either 1 or 2")
+ if width_type not in {"h", "q", "o", "s", "k"}:
+ raise ValueError("width_type must be either h, q, o, s, or k.")
+ if not np.issubdtype(type(width), np.number) or width <= 0:
+ raise ValueError("width should be positive scalar")
+ # TODO: mix does not work for some reason, despite being listed in ffmpeg docs
+ # if mix < 0 or mix > 1:
+ # raise ValueError("mix must be between 0 and 1")
+
+
def low_pass(freq, poles=2, width_type="h", width=.707):
"""
https://ffmpeg.org/ffmpeg-all.html#lowpass
@@ -201,22 +221,12 @@ def low_pass(freq, poles=2, width_type="h", width=.707):
Returns:
filter: A FFmpegFilter object, to be called on an ffmpeg stream
"""
- if not np.issubdtype(type(freq), np.number) or freq <= 0:
- raise ValueError("lowest_freq should be positive scalar")
- if poles not in {1, 2}:
- raise ValueError("poles must be either 1 or 2")
- if width_type not in {"h", "q", "o", "s", "k"}:
- raise ValueError("width_type must be either h, q, o, s, or k.")
- if not np.issubdtype(type(width), np.number) or width <= 0:
- raise ValueError("width should be positive scalar")
- # TODO: mix does not work for some reason, despite being listed in ffmpeg docs
- # if mix < 0 or mix > 1:
- # raise ValueError("mix must be between 0 and 1")
-
+ _pass_arg_check(freq, poles, width_type, width)
return FFmpegFilter("lowpass", f=freq, p=poles,
t=width_type, w=width)
+
def high_pass(freq, poles=2, width_type="h", width=.707):
"""
https://ffmpeg.org/ffmpeg-all.html#highpass
@@ -235,22 +245,11 @@ def high_pass(freq, poles=2, width_type="h", width=.707):
Returns:
filter: A FFmpegFilter object, to be called on an ffmpeg stream
"""
- if not np.issubdtype(type(freq), np.number) or freq <= 0:
- raise ValueError("lowest_freq should be positive scalar")
- if poles not in {1, 2}:
- raise ValueError("poles must be either 1 or 2")
- if width_type not in {"h", "q", "o", "s", "k"}:
- raise ValueError("width_type must be either h, q, o, s, or k.")
- if not np.issubdtype(type(width), np.number) or width <= 0:
- raise ValueError("width should be positive scalar")
-
- # TODO: mix does not work for some reason, despite being listed in ffmpeg docs
- # if mix < 0 or mix > 1:
- # raise ValueError("mix must be between 0 and 1")
-
+ _pass_arg_check(freq, poles, width_type, width)
return FFmpegFilter("highpass", f=freq, p=poles, t=width_type, w=width)
+
def tremolo(mod_freq, mod_depth):
"""
https://ffmpeg.org/ffmpeg-all.html#tremolo
@@ -271,6 +270,7 @@ def tremolo(mod_freq, mod_depth):
return FFmpegFilter("tremolo", f=mod_freq, d=mod_depth)
+
def vibrato(mod_freq, mod_depth):
"""
https://ffmpeg.org/ffmpeg-all.html#vibrato
@@ -291,6 +291,7 @@ def vibrato(mod_freq, mod_depth):
return FFmpegFilter("vibrato", f=mod_freq, d=mod_depth)
+
def chorus(delays, decays, speeds, depths,
in_gain=.4, out_gain=.4):
"""
@@ -299,12 +300,12 @@ def chorus(delays, decays, speeds, depths,
Creates a FFmpegFilter object, when called on an ffmpeg stream,
applies a vibrato filter to the audio signal
Args:
- input_gain: Proportion of input gain
- output_gain: Proportion of output gain
delays: list of delays in ms. Typical Delay is 40ms-6ms
decays: list of decays. Must be between 0 and 1
speeds: list of speeds. Must be between 0 and 1
depths: list of depths. Must be between 0 and 1
+ in_gain: Proportion of input gain
+ out_gain: Proportion of output gain
Returns:
filter: A FFmpegFilter object, to be called on an ffmpeg stream
"""
@@ -330,6 +331,7 @@ def chorus(delays, decays, speeds, depths,
return FFmpegFilter("chorus", in_gain=in_gain,
out_gain=out_gain, delays=delays, speeds=speeds, decays=decays, depths=depths)
+
def phaser(in_gain=.4, out_gain=.74, delay=3,
decay=.4, speed=.5, _type="triangular"):
"""
@@ -338,16 +340,23 @@ def phaser(in_gain=.4, out_gain=.74, delay=3,
Creates a FFmpegFilter object, when called on an ffmpeg stream,
applies a phaser filter to the audio signal
Args:
- input_gain: Proportion of input gain. Must be between 0 and 1
- output_gain: Proportion of output gain. Must be between 0 and 1.
+ in_gain: Proportion of input gain. Must be between 0 and 1
+ out_gain: Proportion of output gain. Must be between 0 and 1.
delay: Delay of chorus filter in ms. (Time between original signal and delayed)
+ decay: Decay of copied signal. Must be between 0 and 1.
speed: Modulation speed of the delayed filter.
- _type: modulation type
+ _type: modulation type. Either Triangular or Sinusoidal
+ "triangular" or "t" for Triangular
+ "sinusoidal" of "s" for sinusoidal
Returns:
filter: A FFmpegFilter object, to be called on an ffmpeg stream
"""
- if (in_gain > 1 or in_gain < 0 or out_gain > 1 or out_gain < 0):
- raise ValueError("in_gain and out_gain must be between 0 and 1.")
+ if in_gain > 1 or in_gain < 0:
+ raise ValueError("in_gain must be between 0 and 1.")
+ if out_gain > 1 or out_gain < 1:
+ raise ValueError("out_gain must be between 0 and 1.")
+ if delay > 1 or delay < 1:
+ raise ValueError("delay must be between 0 and 1.")
allowed_mod_types = {"triangular", "sinusoidal", "t", "s"}
if _type not in allowed_mod_types:
@@ -364,7 +373,6 @@ def phaser(in_gain=.4, out_gain=.74, delay=3,
def _flanger_argcheck(delay, depth, regen, width,
speed, phase, shape, interp):
-
allowed_shape_types = {"triangular", "sinusoidal"}
if shape not in allowed_shape_types:
raise ValueError(f"shape must be one of the following:\n{allowed_shape_types}")
@@ -390,6 +398,7 @@ def _flanger_argcheck(delay, depth, regen, width,
".1 < speed < 10\n"
)
+
def flanger(delay=0, depth=2, regen=0, width=71,
speed=.5, phase=25, shape="sinusoidal", interp="linear"):
"""
@@ -419,6 +428,7 @@ def flanger(delay=0, depth=2, regen=0, width=71,
depth=depth, regen=regen, width=width, speed=speed, phase=phase, shape=shape,
interp=interp)
+
def emphasis(level_in, level_out, _type="col", mode='production'):
"""
https://ffmpeg.org/ffmpeg-all.html#aemphasis
@@ -453,10 +463,10 @@ def emphasis(level_in, level_out, _type="col", mode='production'):
return FFmpegFilter("aemphasis", level_in=level_in, level_out=level_out, mode=mode, **type_kwarg)
+
def _compressor_argcheck(level_in, mode, reduction_ratio,
attack, release, makeup, knee, link,
detection, mix, threshold):
-
# The following values are taken from ffmpeg documentation
if (level_in < LEVEL_MIN or level_in > LEVEL_MAX or
mode not in {"upward", "downward"} or
@@ -493,6 +503,7 @@ def _compressor_argcheck(level_in, mode, reduction_ratio,
" .000097563 < threshold < 1\n"
)
+
def compressor(level_in, mode="downward", reduction_ratio=2,
attack=20, release=250, makeup=1, knee=2.8284, link="average",
detection="rms", mix=1, threshold=.125):
@@ -538,7 +549,6 @@ def equalizer(bands):
Creates a FFmpegFilter object, when called on an ffmpeg stream,
applies a equalizer filter to the audio signal.
Args:
- audio_signal: An AudioSignal object
bands: A list of dictionaries, for each band. The required values for each dictionary:
'chn': List of channel numbers to apply filter. Must be list of ints.
'f': central freqency of band

diff:
@@ -327,7 +327,6 @@ class BaseDataset(Dataset):
else:
self.num_channels = audio_signal.num_channels
-
class DataSetException(Exception):
"""
Exception class for errors when working with data sets in nussl.

diff:
@@ -271,6 +271,10 @@ def test_misc_param_check():
with pytest.raises(ValueError):
effects.phaser(in_gain=-1)
+ with pytest.raises(ValueError):
+ effects.phaser(out_gain=-1)
+ with pytest.raises(ValueError):
+ effects.phaser(decay=-1)
with pytest.raises(ValueError):
effects.phaser(_type="fail")

original_message: cleaned up for pep8