Brazil Capture Station
Solved by the Brazilian Pipeline Team
Task
Can you help establish a Red Hen capture station in Brazil?
If so, write to
and we will try to connect you with a mentor.
Related links
Linux resources
Support for the Brazilian ISDB-T television standard, which is similar to the standard used in Japan, has been added to the Linux kernel's DVB subsystem by Mauro Carvalho Chehab, Anshul Maheswari added ISDB subtitle support to CCExtractor in 2015, sponsored by a team of Brazilian scholars and Red Hen.
Silicon Dust released HDHomeRun for ISDB-T in December 2016; Red Hen switched to using this device, since it makes the operation of the Brazilian capture station identical to our other capture stations in the US and Europe.
The information about the Pixelview Play TV tuner below is left for historical purposes only and is not the recommended procedure.
Pixelview Play TV
The USB capture device PixelView Play TV USB SBTVD Full Seg (HDTV for Brazil), available from ProLink, works well in a Linux capture machine. While the PixelView is well-supported in Linux (using the dvb-usb-dib0700 kernel module, cf. CONFIG_DVB_USB_DIB0700; see instructions), we still need to verify that we can get it working.
A promising candidate for a Linux capture station the Raspberry Pi 3. For details, see How to set up a Red Hen capture station.
Note that the Raspberry Pi 3 has a power supply limit for the USB ports that appears to be 600mA (unverified). If the device consumption is higher than the limit, the device won't work. We do not yet know the power requirements of the PixelView PlayTV USB SBTVD. Other USB capture devices have power requirements as low as 250mA. It's possible we need a powered USB hub for the PixelView.
For CCExtractor, we need version 0.78 or later; see How to install CCExtractor.
Configuration
Add FQDN esfinge.virtua.com.br to /etc/hostname and /etc/mailname and /etc/hosts.
apt-get install netselect-apt wajig
cd /usr/bin ; ln -s wajig just
Run netselect-apt to identify the best local repository and add it to /etc/apt/sources.list.
Add stable-backport and deb-multimedia repositories.
just install dvb-apps screen dvbsnoop tightvncserver strace vlc mplayer
Kernel module
The driver for the PixelView PlayTV USB SBTVD capture device is the kernel module dvb-usb-dib0700. The firmware is available through firmware-linux-nonfree. The module didn't load on boot, so I issued:
modprobe dvb-usb-dib0700
and got this response (the middle line is from before we installed the firmware):
[ 14.196753] dvb-usb: found a 'Prolink Pixelview SBTVD' in cold state, will try to load a firmware
[ 14.221158] usb 5-1: firmware: failed to load dvb-usb-dib0700-1.20.fw (-2)
[ 14.223825] usbcore: registered new interface driver dvb_usb_dib0700
On a fresh reboot, we see the device load the firmware and become active:
$ grep dvb 2016-01-15_dmesg-esfinge-02
[ 14.214329] dvb-usb: found a 'Prolink Pixelview SBTVD' in cold state, will try to load a firmware
[ 14.295585] usb 5-1: firmware: direct-loading firmware dvb-usb-dib0700-1.20.fw
[ 15.000490] dvb-usb: found a 'Prolink Pixelview SBTVD' in warm state.
[ 15.000663] dvb-usb: will pass the complete MPEG2 transport stream to the software demuxer.
[ 15.542172] dvb-usb: schedule remote query interval to 50 msecs.
[ 15.542180] dvb-usb: Prolink Pixelview SBTVD successfully initialized and connected.
[ 15.544161] usbcore: registered new interface driver dvb_usb_dib0700
Configure the channels
See blog guide for Brazil and general guide from linuxtv.
Get the Brazilian ISDB-T frequency table and save it as FrequencyTable. Issue
scan FrequencyTable >channels.conf
On esfinge, we're picking up these channels:
RCI_Movel:473142857:INVERSION_AUTO:BANDWIDTH_6_MHZ:FEC_AUTO:FEC_AUTO:QAM_AUTO:
TRANSMISSION_MODE_AUTO:GUARD_INTERVAL_AUTO:HIERARCHY_NONE:529:530:32024
RCI_HD:473142857:INVERSION_AUTO:BANDWIDTH_6_MHZ:FEC_AUTO:FEC_AUTO:QAM_AUTO:
TRANSMISSION_MODE_AUTO:GUARD_INTERVAL_AUTO:HIERARCHY_NONE:711:731:32000
[bde0]:515142857:INVERSION_AUTO:BANDWIDTH_6_MHZ:FEC_AUTO:FEC_AUTO:QAM_AUTO:
TRANSMISSION_MODE_AUTO:GUARD_INTERVAL_AUTO:HIERARCHY_NONE:97:99:48608
[bdf8]:515142857:INVERSION_AUTO:BANDWIDTH_6_MHZ:FEC_AUTO:FEC_AUTO:QAM_AUTO:
TRANSMISSION_MODE_AUTO:GUARD_INTERVAL_AUTO:HIERARCHY_NONE:0:0:48632
[ebc0]:521142857:INVERSION_AUTO:BANDWIDTH_6_MHZ:FEC_AUTO:FEC_AUTO:QAM_AUTO:
TRANSMISSION_MODE_AUTO:GUARD_INTERVAL_AUTO:HIERARCHY_NONE:501:511:60352
[ebd8]:521142857:INVERSION_AUTO:BANDWIDTH_6_MHZ:FEC_AUTO:FEC_AUTO:QAM_AUTO:
TRANSMISSION_MODE_AUTO:GUARD_INTERVAL_AUTO:HIERARCHY_NONE:0:0:60376
[bd00]:533142857:INVERSION_AUTO:BANDWIDTH_6_MHZ:FEC_AUTO:FEC_AUTO:QAM_AUTO:
TRANSMISSION_MODE_AUTO:GUARD_INTERVAL_AUTO:HIERARCHY_NONE:513:514:48384
[bd18]:533142857:INVERSION_AUTO:BANDWIDTH_6_MHZ:FEC_AUTO:FEC_AUTO:QAM_AUTO:
TRANSMISSION_MODE_AUTO:GUARD_INTERVAL_AUTO:HIERARCHY_NONE:0:0:48408
[bdc0]:551142857:INVERSION_AUTO:BANDWIDTH_6_MHZ:FEC_AUTO:FEC_AUTO:QAM_AUTO:
TRANSMISSION_MODE_AUTO:GUARD_INTERVAL_AUTO:HIERARCHY_NONE:233:234:48576
[bdd9]:551142857:INVERSION_AUTO:BANDWIDTH_6_MHZ:FEC_AUTO:FEC_AUTO:QAM_AUTO:
TRANSMISSION_MODE_AUTO:GUARD_INTERVAL_AUTO:HIERARCHY_NONE:0:0:48601
[bce0]:563142857:INVERSION_AUTO:BANDWIDTH_6_MHZ:FEC_AUTO:FEC_AUTO:QAM_AUTO:
TRANSMISSION_MODE_AUTO:GUARD_INTERVAL_AUTO:HIERARCHY_NONE:273:274:48352
[bcf8]:563142857:INVERSION_AUTO:BANDWIDTH_6_MHZ:FEC_AUTO:FEC_AUTO:QAM_AUTO:
TRANSMISSION_MODE_AUTO:GUARD_INTERVAL_AUTO:HIERARCHY_NONE:0:0:48376
[bd20]:623142857:INVERSION_AUTO:BANDWIDTH_6_MHZ:FEC_AUTO:FEC_AUTO:QAM_AUTO:
TRANSMISSION_MODE_AUTO:GUARD_INTERVAL_AUTO:HIERARCHY_NONE:273:274:48416
[bd38]:623142857:INVERSION_AUTO:BANDWIDTH_6_MHZ:FEC_AUTO:FEC_AUTO:QAM_AUTO:
TRANSMISSION_MODE_AUTO:GUARD_INTERVAL_AUTO:HIERARCHY_NONE:0:0:48440
NGT Digital:659142857:INVERSION_AUTO:BANDWIDTH_6_MHZ:FEC_AUTO:FEC_AUTO:QAM_AUTO:
TRANSMISSION_MODE_AUTO:GUARD_INTERVAL_AUTO:HIERARCHY_NONE:110:111:48992
NGT 1SEG:659142857:INVERSION_AUTO:BANDWIDTH_6_MHZ:FEC_AUTO:FEC_AUTO:QAM_AUTO:
TRANSMISSION_MODE_AUTO:GUARD_INTERVAL_AUTO:HIERARCHY_NONE:210:211:49017
[ee20]:665142857:INVERSION_AUTO:BANDWIDTH_6_MHZ:FEC_AUTO:FEC_AUTO:QAM_AUTO:
TRANSMISSION_MODE_AUTO:GUARD_INTERVAL_AUTO:HIERARCHY_NONE:33:34:60960
[ee38]:665142857:INVERSION_AUTO:BANDWIDTH_6_MHZ:FEC_AUTO:FEC_AUTO:QAM_AUTO:
TRANSMISSION_MODE_AUTO:GUARD_INTERVAL_AUTO:HIERARCHY_NONE:0:0:60984
[ea60]:671142857:INVERSION_AUTO:BANDWIDTH_6_MHZ:FEC_AUTO:FEC_AUTO:QAM_AUTO:
TRANSMISSION_MODE_AUTO:GUARD_INTERVAL_AUTO:HIERARCHY_NONE:258:259:60000
[ea78]:671142857:INVERSION_AUTO:BANDWIDTH_6_MHZ:FEC_AUTO:FEC_AUTO:QAM_AUTO:
TRANSMISSION_MODE_AUTO:GUARD_INTERVAL_AUTO:HIERARCHY_NONE:0:0:60024
[ea7e]:671142857:INVERSION_AUTO:BANDWIDTH_6_MHZ:FEC_AUTO:FEC_AUTO:QAM_AUTO:
TRANSMISSION_MODE_AUTO:GUARD_INTERVAL_AUTO:HIERARCHY_NONE:0:0:60030
[ea7f]:671142857:INVERSION_AUTO:BANDWIDTH_6_MHZ:FEC_AUTO:FEC_AUTO:QAM_AUTO:
TRANSMISSION_MODE_AUTO:GUARD_INTERVAL_AUTO:HIERARCHY_NONE:0:0:60031
[e858]:707142857:INVERSION_AUTO:BANDWIDTH_6_MHZ:FEC_AUTO:FEC_AUTO:QAM_AUTO:
TRANSMISSION_MODE_AUTO:GUARD_INTERVAL_AUTO:HIERARCHY_NONE:0:0:59480
[e840]:707142857:INVERSION_AUTO:BANDWIDTH_6_MHZ:FEC_AUTO:FEC_AUTO:QAM_AUTO:
TRANSMISSION_MODE_AUTO:GUARD_INTERVAL_AUTO:HIERARCHY_NONE:273:274:59456
The logical device for the adapter is /dev/dvb/adapter0/dvr0:
crw-rw----+ 1 root video 212, 1 Jan 15 23:45 /dev/dvb/adapter0/dvr0
Give user csa access (we may not need all of these):
adduser csa video audio plugdev dip cdrom netdev
chmod g+rw /dev/dvb/adapter0/dvr0
This is now working -- user csa has access to the adapter.
Make a recording
The channel script now works, so you can use
channel 4, 38min, "Jornal Hoje", 1, 48352, "Morning news and special reports"
Record to file straight with gnutv:
CH=NGT_Digital
gnutv -out file $CH.mpg -timeout 100 -channels channels.conf $CH
Using frontend "DiBcom 8000 ISDB-T", type DVB-T
status S VYL | signal 4da2 | snr 0064 | ber 00000140 | unc 000017df | FE_HAS_LOCK
Try all the channels:
for CH in `cat channels.conf | cut -d: -f1` ; do
gnutv -out file $(date +%F_%H%M)_BR_"$CH"_test.mpg -timeout 100 -channels channels.conf $CH
done
The bd20 (Globo) channel works!
Record half an hour:
CH=bd20
gnutv -out file $(date +%F_%H%M)_BR_"$CH"_test.mpg -timeout 1800 -channels channels.conf $CH
Here are the various streams according to ffprobe -- note the program number, "Program 48352":
Input #0, mpegts, from 'globo1.mpg':
Duration: 00:01:59.73, start: 31897.135756, bitrate: 15775 kb/s
Program 48352
Stream #0:0[0x111]: Video: h264 (High) ([27][0][0][0] / 0x001B), yuv420p(tv, bt709), 1920x1080 [SAR 1:1 DAR 16:9], 29.97 fps, 29.97 tbr, 90k tbn, 59.94 tbc
Stream #0:1[0x112](por): Audio: aac_latm (HE-AAC) ([17][0][0][0] / 0x0011), 48000 Hz, stereo, fltp
Stream #0:2[0x113](eng): Audio: aac_latm (HE-AAC) ([17][0][0][0] / 0x0011), 48000 Hz, 5.1, fltp
Stream #0:3[0x116]: Unknown: none ([6][0][0][0] / 0x0006)
Stream #0:4[0x384]: Unknown: none ([11][0][0][0] / 0x000B)
Stream #0:5[0x1f4]: Unknown: none ([5][0][0][0] / 0x0005)
Stream #0:6[0x114](por): Audio: aac_latm (HE-AAC) ([17][0][0][0] / 0x0011), 48000 Hz, stereo, fltp
Stream #0:7[0x115](por): Audio: aac_latm (HE-AAC) ([17][0][0][0] / 0x0011), 48000 Hz, 5.1, fltp
The claim that stream 113 is English audio is likely not correct, but it's worth checking. The stream mapping for Globo should be "FFMAP="-map 0:0 -map 0:1" HBMAP="-a 1".
Extracting the text
We use ccextractor-0.82 (with ISDB timestamp fixes) to extract the text, first in debug mode:
csa@esfinge:~$ ccextractor-0.82 -debug -ts -noru -out=ttxt -utf8 -o globo1.ccx.out globo1.mpg
CCExtractor 0.82, Carlos Fernandez Sanz, Volker Quetschke.
Teletext portions taken from Petr Kutalek's telxcc
--------------------------------------------------------------------------
Input: globo1.mpg
[Extract: 1] [Stream mode: Transport]
[Program : Auto ] [Hauppage mode: No] [Use MythTV code: Auto]
[Timing mode: Auto] [Debug: Yes] [Buffer input: No]
[Use pic_order_cnt_lsb for H.264: No] [Print CC decoder traces: No]
[Target format: (null)] [Encoding: UTF-8] [Delay: 0] [Trim lines: No]
[Add font color data: Yes] [Add font typesetting: Yes]
[Convert case: No] [Video-edit join: No]
[Extraction start time: not set (from start)]
[Extraction end time: not set (to end)]
[Live stream: No] [Clock frequency: 90000]
[Teletext page: Autodetect]
[Start credits text: None]
-----------------------------------------------------------------
Opening file: globo1.mpg
Analyzing data in general mode
This TS file has more than one program. These are the program numbers found:
48352
48376
Program map section (PMT)
H.264 video stream [0x1b] - PID: 273
Unknown stream [0x11] - PID: 274
Unknown stream [0x11] - PID: 275
*****ISDB subtitles detected
MPEG-2 private data stream [0x06] - PID: 278
ISO/IEC 13818-6 type B stream [0x0b] - PID: 900
MPEG-2 private table sections stream [0x05] - PID: 500
Unknown stream [0x11] - PID: 276
Unknown stream [0x11] - PID: 277
So the ISDB subtitles are detected; note the two numbers above -- 48352 and 48376.
It turns out it's the first program number that contains the text (as ffprobe tells us above):
csa@esfinge:~$ ccextractor-0.82 -debug -ts -pn 48352 -noru -out=ttxt -utf8 -o globo1.ccx.out globo1.mpg
When we cat the output or read it in nano, the characters don't display correctly, but vi shows they are in fact correct:
20161114152002.000|20161114152012.699|CC1|>> [ZILEIDE SILVA] BOA TARDE.
20161114152002.000|20161114152012.699|CC1|UMA VISÃO ÚNICA EM 68 ANOS. A
20161114152002.000|20161114152012.699|CC1|>> [ZILEIDE SILVA] BOA TARDE.
20161114152012.699|20161114152015.764|CC1|MAIOR SUPER LUA DAS ÚLTIMAS
20161114152015.764|20161114152016.458|CC1|DÉCADAS ILUMINA O CÉU PELO MUNDO
20161114152016.458|20161114152019.292|CC1|E PODERÁ SER VISTA HOJE À NOITE
20161114152019.292|20161114152019.870|CC1|AQUI NO BRASIL. O CHÃO VOLTA A
20161114152019.870|20161114152021.663|CC1|TREMER NA NOVA ZELÂNDIA DEPOIS
20161114152021.663|20161114152023.398|CC1|DO TERREMOTO QUE MATOU DUAS
20161114152023.398|20161114152025.191|CC1|PESSOAS NO FIM DE SEMANA. A ONU
20161114152025.191|20161114152026.984|CC1|PREVÊ UM CALOR RECORDE PARA ESSE
20161114152026.984|20161114152027.909|CC1|ANO NO PLANETA. O PRESIDENTE
20161114152027.909|20161114152030.222|CC1|ELEITO DONALD TRUMP ANUNCIA DOIS
20161114152030.222|20161114152031.784|CC1|NOMES DA FUTURA EQUIPE DE
20161114152031.784|20161114152032.709|CC1|GOVERNO. E AQUI NO BRASIL,
20161114152032.709|20161114152035.081|CC1|ECONOMISTAS APOSTAM NUMA QUEDA
20161114152035.081|20161114152036.816|CC1|MAIS LENTA DO JUROS DEPOIS DA
20161114152036.816|20161114152039.649|CC1|ELEIÇÃO DE TRUMP. DISPARA O
20161114152039.649|20161114152040.922|CC1|NÚMERO DE PESSOAS QUE NÃO PAGAM
20161114152040.922|20161114152041.963|CC1|A CONTA DE LUZ E TEM A ENERGIA
20161114152041.963|20161114152045.549|CC1|CORTADA MAS A GENTE EXPLICA QUE
20161114152045.549|20161114152046.821|CC1|DÁ PARA PARCELAR A FATURA. E
20161114152046.821|20161114152048.787|CC1|MESMO COM A CRISE, O MERCADO DE
20161114152048.787|20161114152049.481|CC1|CARTÃO DE CRÉDITO NÃO PARA DE
20161114152049.481|20161114152051.274|CC1|CRESCER. O PERIGO ESTÁ NOS
20161114152051.274|20161114152055.033|CC1|JUROS. O MAIOR AVIÃO DO MUNDO
20161114152055.033|20161114152056.074|CC1|POUSA NO BRASIL E ATRAI UMA
20161114152056.074|20161114152057.636|CC1|MULTIDÃO DE CURIOSOS. E O JOGO
20161114152057.636|20161114152058.792|CC1|COMUM ENTRE JOVENS E
20161114152058.792|20161114152102.725|CC1|ADOLESCENTES GANHA ESPAÇO NO
20161114152102.725|20161114152110.012|CC1|TREINAMENTO DE EXECUTIVOS. O
20161114152110.012|20161114152111.979|CC1|JORNAL HOJE ESTÁ COMEÇANDO.