opensourcemirror
/
adblockradio
mirror of https://github.com/adblockradio/adblockradio.git
1
0
Fork 0
Code Issues Projects Releases Wiki Activity
adblockradio/README.md

288 lines
12 KiB
Markdown
Raw Permalink Blame History

# Adblock Radio (project archived)
![Adblock Radio](https://www.adblockradio.com/assets/img/abr_buddha_v3_175.png)
A library to block ads on live radio streams and podcasts. Machine learning meets Shazam.
Engine of [AdblockRadio.com](https://www.adblockradio.com).
Demo standalone player [available here](https://github.com/adblockradio/buffer-player).
Build status:
[![CircleCI](https://circleci.com/gh/adblockradio/adblockradio.svg?style=svg)](https://circleci.com/gh/adblockradio/adblockradio)
Help the project grow:
<a href="https://liberapay.com/asto/donate"><img alt="Donate using Liberapay" src="https://liberapay.com/assets/widgets/donate.svg"></a>
## Overview
A technical discussion is available [here](https://www.adblockradio.com/blog/2018/11/15/designing-audio-ad-block-radio-podcast/).
Radio streams are downloaded in `predictor.js` with the module [adblockradio/stream-tireless-baler](https://github.com/adblockradio/stream-tireless-baler). Podcasts are downloaded in `predictor-file.js`.
In both cases, audio is then decoded to single-channel, `22050 Hz` PCM with `ffmpeg`.
Chunks of about one second of PCM audio are piped into two sub-modules:
- a time-frequency analyser (`predictor-ml/ml.js`), that analyses spectral content with a neural network.
- a fingerprint matcher (`predictor-db/hotlist.js`), that searches for exact occurrences of known ads, musics or jingles.
In `post-processing.js`, results are gathered for each audio segment and cleaned.
A Readable interface, `Analyser`, is exposed to the end user. It streams objects containing the audio itself and all analysis results.
On a regular laptop CPU and with the Python time-frequency analyser, computations run at 5-10X for files and at 10-20% usage for live stream.
## Getting started
### Installation
##### Mandatory prerequisites:
You need Node.js (>= v10.12.x, but < 11) and NPM. Download it [here](https://nodejs.org/en/download/). Pro-tip: to manage several node versions on your platform, use [NVM](https://github.com/creationix/nvm).
On Debian Stretch:
```bash
apt-get install -y git ssh tar gzip ca-certificates build-essential sqlite3 ffmpeg
```
Note: works on Jessie, but installing ffmpeg is a bit painful. See [here](https://ffmpeg.org/download.html) and [there](https://superuser.com/questions/286675/how-to-install-ffmpeg-on-debian).
##### Optional prerequisites:
For best performance (~2x speedup) you should choose to do part of the computations with Python. Additional prerequisites are the following: Python (tested with v2.7.9), [Keras](https://keras.io/#installation) (tested with v2.0.8) and [Tensorflow](https://www.tensorflow.org/install/) (tested with CPU v1.4.0 and GPU v1.3.0).
On Debian:
```bash
apt-get install python-dev portaudio19-dev
pip install python_speech_features h5py numpy scipy keras tensorflow zerorpc sounddevice psutil
```
Note: if you do not have pip [follow these instructions to install it](https://pip.pypa.io/en/stable/installing/).
##### Then install this module:
```bash
git clone https://github.com/adblockradio/adblockradio.git
cd adblockradio
npm install
```
### Testing
Validate your installation with the test suite:
```
npm test
```
### Command-line demo
At startup and periodically during runtime, filter configuration files are automatically updated from [adblockradio.com/models/](https://adblockradio.com/models/):
- a compatible machine-learning model (`model.keras` or `model.json` + `group1-shard1of1`), for the time-frequency analyser.
- a fingerprint database (`hotlist.sqlite`), for the fingerprint matcher.
#### Live stream analysis
Run the demo on French RTL live radio stream:
```bash
node demo.js
```
Here is a sample output of the demo script, showing an ad detected:
```
{
"gain": 74.63,
"ml": {
"class": "0-ads",
"softmaxraw": [
0.996,
0.004,
0
],
"softmax": [
0.941,
0.02,
0.039
],
"slotsFuture": 4,
"slotsPast": 5
},
"hotlist": {
"class": "9-unsure",
"file": null,
"matches": 1,
"total": 7
},
"class": "0-ads",
"metadata": {
"artist": "Laurent Ruquier",
"title": "L'été des Grosses Têtes",
"cover": "https://cdn-media.rtl.fr/cache/wQofzw9SfgHNHF1rqJA3lQ/60v73-2/online/image/2014/0807/7773631957_laurent-ruquier.jpg"
},
"streamInfo": {
"url": "http://streaming.radio.rtl.fr/rtl-1-44-128",
"favicon": "https://cdn-static.rtl.fr/versions/www/6.0.637/img/apple-touch-icon.png",
"homepage": "http://www.rtl.fr/",
"audioExt": "mp3"
},
"predictorStartTime": 1531150137583,
"playTime": 1531150155250,
"tBuffer": 15.98,
"audio": ...
}
```
#### Podcast analysis
It is also possible to analyse radio recordings.
Run the demo on a recording of French RTL radio, including ads, talk and music:
```bash
node demo-file.js
```
Gradual outputs are similar to those of live stream analysis. An additional post-processing specific to recordings hides the uncertainties in predictions and shows big chunks for each class, with time stamps in milliseconds, making it ready for slicing.
```
[
{
"class": "1-speech",
"tStart": 0,
"tEnd": 58500
},
{
"class": "0-ads",
"tStart": 58500,
"tEnd": 125500
},
{
"class": "1-speech",
"tStart": 125500,
"tEnd": 218000
},
{
"class": "2-music",
"tStart": 218000,
"tEnd": 250500
},
{
"class": "1-speech",
"tStart": 250500,
"tEnd": 472949
}
]
```
Note that when analyzing audio files, you still need to provide the name of a radio stream, because the algorithm has to load acoustic parameters and DB of known samples. Analysis of podcasts not tied to a radio is not yet supported, but may possibly be in the future.
## Documentation
### Usage
Below is a simple usage example. More thorough usage examples are available in the tests:
- file/podcast analysis: `test/file.js`
- live stream analysis: `test/online.js`
- record a live stream, analyse it later: `test/offline.js`
```javascript
const { Analyser } = require("adblockradio");
const abr = new Analyser({
country: "France",
name: "RTL",
config: {
...
}
});
abr.on("data", function(obj) {
...
});
```
Property|Description|Default
--------|-----------|-------
`country`|Country of the radio stream according to [radio-browser.info](http://www.radio-browser.info)|None
`name`|Name of the radio stream according to [radio-browser.info](http://www.radio-browser.info)|None
`file`|File to analyse (optional, analyse the live stream otherwise)|None
### Methods
Acoustic model and hotlist files are refreshed automatically on startup. If you plan to continuously run the algo for a long time, you can trigger manual updates. Note those methods are only available in live stream analysis mode.
Method|Parameters|Description
------|----------|-----------
`refreshPredictorMl`|None|Manually refresh the ML model (live stream only)
`refreshPredictorHotlist`|None|Manually refresh the hotlist DB (live stream only)
`refreshMetadata`|None|Manually refresh the [metadata scraper](https://github.com/adblockradio/webradio-metadata) (live stream only)
`stopDl`|None|Stop Adblock Radio (live stream only)
### Optional configuration
Properties marked with a `*` are meant to be used only with live radio stream analysis, not file analysis where they are ignored.
#### Scheduling
Property|Description|Default
--------|-----------|-------
`predInterval`|Send stream status to listener every N seconds|`1`
`saveDuration*`|If enabled, save audio file and metadata every N `predInterval` times|`10`
`modelUpdatesInterval`|If enabled, update model files every N minutes|`60`
#### Switches
Property|Description|Periodicity|Default
--------|-----------|-----------|-------
`enablePredictorMl`|Perform machine learning inference|`predInterval`|`true`
`JSPredictorMl`|Use tfjs instead of Python for ML inference (slower)|`false`
`enablePredictorHotlist`|Compute audio fingerprints and search them in a DB|`predInterval`|`true`
`saveAudio*`|Save stream audio data in segments on hard drive|`saveDuration`|`true`
`saveMetadata`|Save a JSON with predictions|`saveDuration`|`true`
`fetchMetadata*`|Gather metadata from radio websites|`saveDuration`|`true`
`modelUpdates`|Keep ML and hotlist files up to date|`modelUpdatesInterval`|`true`
#### Paths
Property|Description|Default
--------|-----------|-------
`modelPath`|Directory where ML models and hotlist DBs are stored|`process.cwd() + '/model'`
`modelFile`|Path of ML file relative to `modelPath`|`country + '_' + name + '/model.keras'`
`hotlistFile`|Path of the hotlist DB relative to `modelPath`|`country + '_' + name + '/hotlist.sqlite'`
`saveAudioPath*`|Root folder where audio and metadata are saved|`process.cwd() + '/records'`
### Output
Readable streams constructed with `Analyser` emit objects with the following properties. Some properties are only available when doing live radio analysis. They are marked with a `*`. Other specific to file analysis are marked with `**`.
- `audio*`: Buffer containing a chunk of original (compressed) audio data.
- `ml`: `null` if not available, otherwise an object containing the results of the time-frequency analyser
* `softmaxraw`: an array of three numbers representing the [softmax](https://en.wikipedia.org/wiki/Softmax_function) between ads, speech and music.
* `softmax`: same as softmaxraw, but smoothed in time with `slotsFuture` data points in the future and `slotsPast` data points in the past. Smoothing weights are defined by `consts.MOV_AVG_WEIGHTS` in [`post-processing.js`](https://github.com/adblockradio/adblockradio/blob/master/post-processing.js).
* `class`: either `0-ads`, `1-speech`, `2-music` or `9-unsure`. The classification according to `softmax`.
- `hotlist`: null if not available, otherwise an object containing the results of the fingerprint matcher.
* `file`: if class is not "9-unsure", the reference of the file recognized.
* `total`: number of fingerprints computed for the given audio segment.
* `matches`: number of matching fingerprints between the audio segment and the fingerprint database.
* `class`: either `0-ads`, `1-speech`, `2-music`, `3-jingles` or `9-unsure` if not enough matches have been found.
- `class`: final prediction of the algorithm. Either `0-ads`, `1-speech`, `2-music`, `3-jingles` or `9-unsure`.
- `metadata*`: live metadata, fetched and parsed by the module [adblockradio/webradio-metadata](https://github.com/adblockradio/webradio-metadata).
- `streamInfo*`: static metadata about the stream. Contains stream `url`, `favicon`, `bitrate` in bytes / s, audio files extension `audioExt` (`mp3` or `aac`) and `homepage` URL.
- `gain`: a [dB](https://en.wikipedia.org/wiki/Decibel) value representing the average volume of the stream. Useful if you wish to normalize the playback volume. Calculated by [`mlpredict.py`](https://github.com/adblockradio/adblockradio/blob/master/predictor-ml/mlpredict.py).
- `tBuffer*`: seconds of audio buffer. Calculated by [adblockradio/stream-tireless-baler](https://github.com/adblockradio/stream-tireless-baler).
- `predictorStartTime*`: timestamp of the algorithm startup. Useful to get the uptime.
- `playTime*`: approximate timestamp of when the given audio is to be played. TODO check this.
- `tStart**`: lower boundary of the time interval linked with the prediction (in milliseconds)
- `tEnd**`: upper boundary of the time interval linked with the prediction (in milliseconds)
## Supported radios
The list of supported radios is [available here](https://github.com/adblockradio/available-models).
### Note to developers
Integrations of this module are welcome. Suggestions are available [here](https://www.adblockradio.com/blog/2018/11/15/designing-audio-ad-block-radio-podcast/#product-design).
A standalone demo player for web browsers is [available here](https://github.com/adblockradio/buffer-player).
## License
See LICENSE file.
Your contribution to this project is welcome, but might be subject to a contributor's license agreement.
Reference in New Issue View Git Blame Copy Permalink