Finally, V5 is here, 3x faster, supporting 6000+ languages!

[snakers4]

Finally, V5 is here, 3x faster, supporting 6000+ languages!

Performance and Model Size

• 3x faster inference for TorchScript, 10% faster inference for ONNX;
• Now TorchScript is as fast as ONNX;
• Model size is 2x larger, 2MB vs. 1MB;

Quality

• The VAD supports more than 6,000 languages now;
• Significanly more robust on noisy data;
• Overall 5-7% quality increase on clean data;
• Quality difference for 8 kHz and 16 kHz is negligible now;
• Quality difference for different window sizes is negligible => window size was deprecated;
• Added benchmarks on 9 unique datasets (2 private) and one holistic multi-domain dataset;

Changes and deprecations

• ONNX opset 16;
• window_size_samples is deprecated - now the VAD only works with fixed size window;
• VAD now works with 8 kHz and 16 kHz sample rates, only with fixed 256 and 512 sample windows respectively;
• Slightly changed internal logic, now some context (part of previous chunk) is passed along with the current chunk;
• Sample rates that are a multiple of 16 kHz are still supported;

---

This discussion was created from the release Finally, V5 is here, 3x faster, supporting 6000+ languages!.

[dgoryeo]

🎉 Congratulations on the release of Silero VAD Version 5! 🎉

Finally, V5 is here! This is a much-anticipated milestone and a testament to the hard work and dedication of you, @snakers4, and the team. Keep up the fantastic work!

[hoonlight]

Really awesome!

[RiccardoRomagnoli]

Awesome work!!

[smallbraingames]

I am noticing degraded performance on this model–am I passing stateN correctly (is it equivalent to how "h" and "c" were passed earlier?)

specifically, how to implement this? "now some context (part of previous chunk) is passed along with the current chunk"

[snakers4]

If you are using the ONNX version, then see changes to this function -

silero-vad/utils_vad.py

Lines 9 to 105 in fdbb0a3

	class OnnxWrapper():
	def __init__(self, path, force_onnx_cpu=False):
	import numpy as np
	global np
	import onnxruntime
	opts = onnxruntime.SessionOptions()
	opts.inter_op_num_threads = 1
	opts.intra_op_num_threads = 1
	if force_onnx_cpu and 'CPUExecutionProvider' in onnxruntime.get_available_providers():
	self.session = onnxruntime.InferenceSession(path, providers=['CPUExecutionProvider'], sess_options=opts)
	else:
	self.session = onnxruntime.InferenceSession(path, sess_options=opts)
	self.reset_states()
	self.sample_rates = [8000, 16000]
	def _validate_input(self, x, sr: int):
	if x.dim() == 1:
	x = x.unsqueeze(0)
	if x.dim() > 2:
	raise ValueError(f"Too many dimensions for input audio chunk {x.dim()}")
	if sr != 16000 and (sr % 16000 == 0):
	step = sr // 16000
	x = x[:,::step]
	sr = 16000
	if sr not in self.sample_rates:
	raise ValueError(f"Supported sampling rates: {self.sample_rates} (or multiply of 16000)")
	if sr / x.shape[1] > 31.25:
	raise ValueError("Input audio chunk is too short")
	return x, sr
	def reset_states(self, batch_size=1):
	self._state = torch.zeros((2, batch_size, 128)).float()
	self._context = torch.zeros(0)
	self._last_sr = 0
	self._last_batch_size = 0
	def __call__(self, x, sr: int):
	x, sr = self._validate_input(x, sr)
	num_samples = 512 if sr == 16000 else 256
	if x.shape[-1] != num_samples:
	raise ValueError(f"Provided number of samples is {x.shape[-1]} (Supported values: 256 for 8000 sample rate, 512 for 16000)")
	batch_size = x.shape[0]
	context_size = 64 if sr == 16000 else 32
	if not self._last_batch_size:
	self.reset_states(batch_size)
	if (self._last_sr) and (self._last_sr != sr):
	self.reset_states(batch_size)
	if (self._last_batch_size) and (self._last_batch_size != batch_size):
	self.reset_states(batch_size)
	if not len(self._context):
	self._context = torch.zeros(batch_size, context_size)
	x = torch.cat([self._context, x], dim=1)
	if sr in [8000, 16000]:
	ort_inputs = {'input': x.numpy(), 'state': self._state.numpy(), 'sr': np.array(sr, dtype='int64')}
	ort_outs = self.session.run(None, ort_inputs)
	out, state = ort_outs
	self._state = torch.from_numpy(state)
	else:
	raise ValueError()
	self._context = x[..., -context_size:]
	self._last_sr = sr
	self._last_batch_size = batch_size
	out = torch.from_numpy(out)
	return out
	def audio_forward(self, x, sr: int):
	outs = []
	x, sr = self._validate_input(x, sr)
	self.reset_states()
	num_samples = 512 if sr == 16000 else 256
	if x.shape[1] % num_samples:
	pad_num = num_samples - (x.shape[1] % num_samples)
	x = torch.nn.functional.pad(x, (0, pad_num), 'constant', value=0.0)
	for i in range(0, x.shape[1], num_samples):
	wavs_batch = x[:, i:i+num_samples]
	out_chunk = self.__call__(wavs_batch, sr)
	outs.append(out_chunk)
	stacked = torch.cat(outs, dim=1)
	return stacked.cpu()

For TorchScript version - the interface is hidden inside of the model and the invocation should remain the same.

[Hiniu6996]

Finally, V5 is here, 3x faster, supporting 6000+ languages!

Performance and Model Size

• 3x faster inference for TorchScript, 10% faster inference for ONNX;
• Now TorchScript is as fast as ONNX;
• Model size is 2x larger, 2MB vs. 1MB;

Quality

• The VAD supports more than 6,000 languages now;
• Significanly more robust on noisy data;
• Overall 5-7% quality increase on clean data;
• Quality difference for 8 kHz and 16 kHz is negligible now;
• Quality difference for different window sizes is negligible => window size was deprecated;
• Added benchmarks on 9 unique datasets (2 private) and one holistic multi-domain dataset;

Changes and deprecations

• ONNX opset 16;
• window_size_samples is deprecated - now the VAD only works with fixed size window;
• VAD now works with 8 kHz and 16 kHz sample rates, only with fixed 256 and 512 sample windows respectively;
• Slightly changed internal logic, now some context (part of previous chunk) is passed along with the current chunk;
• Sample rates that are a multiple of 16 kHz are still supported;

---

This discussion was created from the release Finally, V5 is here, 3x faster, supporting 6000+ languages!.

[jnhck]

Thank you for all your work. This is an awesome project!

[ksyntiant]

Hi I just started downloading data mentioned in the datasets section. I downloaded 3 out of 5 datasets and am already at around 1.8TB of data. I am just wondering how much TB of data was v5 model trained on. Is it possible to get more information on your training data other than provided in the datasets page. Thanks :)