Slick capture

Solved by Turner & Steen. We have now shown that a Raspberry Pi 2B with an HDHomeRun on its local network can capture a broadcast, extract its closed captions, and compress the .mpg file to the appropriate .mp4 file. Transferring the derived files to the archive using rsync etc. is trivial.

Capture of broadcasts is performed at UCLA using powerful Linux servers and HDHomeRun tuners. Since 2011, Red Hen has established a number of small, remote capture stations, using a Mac and an Elgato EyeTV Hybrid tuner. For a description of this design, see a How to set up a Red Hen capture station. For pictures of such an installation (in Cleveland), see redhen1. But that installation requires augmentation of the Mac operating system with many macport packages and some configuration at a distance.

Has the future arrived in the form of credit card computers running Linux, such as Raspberry Pi? These computers are very small and very inexpensive. It looks as if we should be able to purchase one, configure it with the right user accounts, load it with all the right Red Hen software, and then ship it to someone with instructions to give it power, ethernet, an external hard disk for storage, a tuner, and a TV broadcast feed. The machine would then start up and do its job with little further configuration (e.g., selection of broadcasts to record) and little local routine maintenance. It might even be that the complete system would be an RPi ($35), a 16 GB microSD card ($15),a 32 GB USB Flash Drive ($15), and an HDHomeRun Dual ($70 on ebay, $120 elsewhere). Add $15 for anything else nice to have but not necessary (e.g., a case, a wifi adaptor). The result is a complete system for $150-$200, not counting power, ethernet connection, or RTF cable carrying the TV broadcast.

Task

Design a capture station based on a credit-card Linux computer, to become the new standard for a miniature international capture station, replacing the Mac-based design. Can it record a broadcast (the transport stream), extract the closed captions, and compress the original file to a suitable .mp4 file? Let's assume for this task that if we can manually get it to achieve this performance, then we can script the performance as a separate task. Let's also assume for this task that scripting the movement of the output files around the Red Hen network is a minor task, for later, because it is something we have already done for all the remote capture stations.

Would you like to accomplish all or part of this task?

If so, write to

and we will try to connect you with a mentor.

Update July 2015. This page will now be used as the basis for establishing a Portugal capture station.

Some initial information

  1. Getting started with Raspberry Pi:
      1. 2016-01-07: UPDATE about ssh: As of the November 2016 release, Raspbian has the SSH server disabled by default. You will have to enable it manually. This is done using raspi-config: Enter sudo raspi-config in the terminal, first select Interfacing options, then navigate to ssh, press Enter and select Enable or disable ssh server. For headless setup, SSH can be enabled by placing a file named 'ssh', without any extension, onto the boot partition of the SD card.
      2. Update 2016-07-04:
        • If you format your SD card with SDformatter.app and use PiFiller to put the raspbian jessie image on it, and insert the card into a Raspberry Pi 3, then you can connect to the command line on the RPi 3 by connecting an ethernet cable from a Mac to the RPi, giving electrical power to the RPi, opening Terminal on the Mac. From the Mac command line, issue
        • ssh pi@raspberrypi.local
        • The password is raspberry.
        • Once you have connected to the command line on the RPi, configure everything, beginning by changing the password for user pi.
        1. http://www.howtogeek.com/138281/the-htg-guide-to-getting-started-with-raspberry-pi/
    1. How to configure Raspberry Pi for Remote Shell, Desktop, and File Transfer
      1. https://www.raspberrypi.org/documentation/remote-access/ssh/unix.md
      2. https://www.raspberrypi.org/documentation/remote-access/vnc/mac.md
      3. http://www.howtogeek.com/141157/how-to-configure-your-raspberry-pi-for-remote-shell-desktop-and-file-transfer/all/
  2. Debian Linux on Raspberry Pi: https://wiki.debian.org/RaspberryPi
    1. MicroSD card for Raspberry Pi 2 B: http://elinux.org/RPi_SD_cards
    2. (A mere frill: Wifi. How to add wifi to the Raspberry Pi: http://raspberrypihq.com/how-to-add-wifi-to-the-raspberry-pi/ . The instructions on that site work for a private network, but to connect to Wifi with WPA2 Enterprise (e.g., at a university), see http://netbeez.net/2014/10/14/connect-your-raspberry-pi-to-wireless-enterprise-environments-with-wpa-supplicant/ . Given Pi Finder (see below), this is not necessary at all, and certainly not after configuration and installation as capture station, but it might be convenient in some environments during setup. Possible wifi USB adapters:
    3. http://elinux.org/RPi_USB_Wi-Fi_Adapters
      1. http://thepihut.com/products/usb-wifi-adapter-for-the-raspberry-pi
      2. http://www.amazon.com/gp/product/B003MTTJOY/ref=as_li_tl?ie=UTF8&camp=1789&creative=390957&creativeASIN=B003MTTJOY&linkCode=as2&tag=rapihq-20&linkId=7QAXDGG72H4EWYFB )
    4. Developer machine: The ODROID-C2 is a candidate if you need a Linux server for development; it's similar to the RPi 3B but has a faster CPU, more RAM, and hardware decoding of h264, so better for playing videos.

Turner Development Raspberry Pi 2 B

  1. Turner configured the following development machine:
    1. Equipment
      1. Raspberry Pi 2 B, 1GB RAM, 900MHz Quad Core CPU. http://www.amazon.com/gp/product/B00WULV84Y?psc=1&redirect=true&ref_=oh_aui_detailpage_o00_s01
      2. CanaKit Raspberry Pi Micro USB Power Supply / Adapter / Charger
      3. SanDisk Ultra 32GB UHS-I/Class 10 Micro SDHC Memory Card Up to 48MB/s With Adapter- SDSDQUAN-032G-G4A [Newest Version]
      4. Just because they came with the Raspberry Pi 2 B as freebies: Clear Transparent Case and 802.11n Wifi Adapter (RT5370 CHIP). These seem to be unnecessary for our project.
      5. Western Digitial My Book for Mac 1 TB USB 2.0 Desktop External Hard Drive, Model Number WDBAAG0010HCH-NESN. Note that it comes with its own independent power supply: 12 Volts, 1.5 Amps, 18 Watts. Such a large external drive cannot be powered through the RPi USB2 ports. (A clean hard disk formatted for Linux or Windows would have worked better.)
      6. HDHomeRun Dual (US version). This is the networked TV tuner.
      7. Pix:
    1. Formatting the microSD card and installing the operating system on it (using a MacBook Air):
      1. Insert SanDisk microSD into the adapter it came with. Check that the lock switch is off. Insert the adapter into the SDcard slot on MacBook Air. Format the microSD card using the SD Association's Formatting Tool from https://www.sdcard.org/downloads/formatter_4/eula_mac/
      2. Various possibilities for obtaining and installing the operating system
        1. Update 2016-10-16. We no longer make disk images of existing capture stations. Instead, download the latest Raspbian (currently Debian Buster) from https://www.raspberrypi.org/downloads/raspbian/ and install it on the formatted microSD card with something like Pi Filler from http://ivanx.com/raspberrypi/. Doing so will make your Raspberry Pi's system up to date. We then build up the additional Red Hen Software from /nest. For information about /nest, see https://sites.google.com/site/distributedlittleredhen/home/the-cognitive-core-research-topics-in-red-hen/the-barnyard/completed-barnyard-projects/re-design-the-process-for-installing-a-capture-station-convert-cwru-to-the-new-design . UPDATE 2015-06-23: If someone solves the task Scripted Slick Capture, then we could make an image of an SD card already containing not only the system but also the relevant Red Hen software and make that image available via internet. Accordingly, the local capture site could purchase all the equipment and receive the image to install on the SD card, meaning that there would be no need to ship anything physically from Red Hen to the remote capturing site.]
        2. Download NOOBS (offline and network install version) from https://www.raspberrypi.org/downloads/ . NOOBS is an easy operating system installer which includes Raspian [Debian Linux optimized for RPi] and a selection of operating systems which are then downloaded from the internet for installation. Follow instructions in the INSTRUCTIONS-readme.txt inside the downloaded and decompressed NOOBS folder.
        3. Update 2017-08-08. We now use dd on a Mac in preference to PiFiller. See instructions at http://www.runeaudio.com/documentation/quick-start/sd-card-setup-mac/. 2015-05-29. This is the method selected: Download Raspian from https://www.raspberrypi.org/downloads/. Turner downloaded 2015-05-05-raspbian-wheezy.zip. Decompress the .zip file into the .img file. Download PiFiller and PiFinder from http://ivanx.com/raspberrypi/ and follow the instructions for PiFiller on that page. That is, just click on the PiFiller.app and do what it says.
        4. For later, here's a way to make an image of the microSD card: PiCopier (http://ivanx.com/raspberrypi/) offers to make a snapshot of an SD card—a kind of PiFiller in reverse.
    2. Connecting to the RPi: PiFinder (http://ivanx.com/raspberrypi/) offers to determine the IP address of your RPi on your network, so you can connect to it via a terminal window, and also to find the MAC address of the RPi, so it can be added to the router as a DHCP reservation, with the result that the RPi will be assigned the same IP address every time and you won't need to run PiFinder again.
    1. Configuring users. User management in Raspbian is done on the command line. The default user is pi with the password raspberry. You can add users and change each user's password. See https://www.raspberrypi.org/documentation/linux/usage/users.md
    1. Configure raspbian with
      1. $ sudo raspi-config
      2. See http://elinux.org/RPi_raspi-config . Expand filesystem. Use Internationalization Options to change locale & timezone: e.g., for installation of redhen3rpi in Cleveland, Ohio, deselect en_GB.UTF-8 UTF-8 and select en_US.UTF8 UTF-8; set timezone to US ET. Use Advanced Options to set hostname (e.g. redhen3rpi). Reboot.
      3. Update. (First update existing distribution as below, then edit /etc/apt/sources.list to change to the distribution you want, e.g. deb http://mirrordirector.raspbian.org/raspbian/ jessie main contrib non-free rpi, then update by going selectively through the list again. )
        1. $ sudo apt-get update
        2. $ sudo apt-get upgrade
        3. $ sudo shutdown -r now
        4. [now log back in]
        5. $ sudo apt-get install ca-certificates
        6. $ sudo apt-get distupgrade
        7. $ sudo apt-get install git-core
        8. $ sudo apt-get install rpi-update
        9. $ sudo rpi-update
        10. $ sudo apt-get install eject
        11. $ sudo shutdown -r now
        12. [now log back in]
        13. Change default editor to vi
        14. $ sudo update-alternatives --set editor /usr/bin/vim.tiny
        15. Generate RSA keys. ssh-keygen -t rsa. Put them wherever needed on other systems.
        16. Optional: add wifi adapter, just because we happen to have one. Ours is labeled 802.11n. Plug it into a USB port on RPi. Follow instructions at https://www.raspberrypi.org/documentation/configuration/wireless/wireless-cli.md . Ours works, as we see from the existence of an inet address:
          1. mark@redhen3rpi ~ $ ifconfig wlan0
          2. wlan0 Link encap:Ethernet HWaddr 00:13:ef:c0:05:28
          3. inet addr:10.0.1.131 Bcast:10.0.1.255 Mask:255.255.255.0
    2. Can we add an external hard drive? Yes. Here is how we added a USB Verbatim 8GB thumbdrive. First, insert the thumb drive into a USB port on the RPi.
      1. See http://superuser.com/questions/662614/raspberry-pi-how-to-format-hdd and http://www.itechlounge.net/2012/01/linux-partition-and-format-external-hard-drive-as-ext3-filesystem/
      2. mark@redhen3rpi ~ $ dmesg | tail -20
      3. [31247.412404] usb 1-1.4: new high-speed USB device number 82 using dwc_otg
      4. [31247.517453] usb 1-1.4: New USB device found, idVendor=18a5, idProduct=0302
      5. [31247.517480] usb 1-1.4: New USB device strings: Mfr=1, Product=2, SerialNumber=3
      6. [31247.517498] usb 1-1.4: Product: STORE N GO
      7. [31247.517514] usb 1-1.4: Manufacturer: Verbatim
      8. [31247.517531] usb 1-1.4: SerialNumber: 070141FE0CC22009
      9. [31247.518728] usb-storage 1-1.4:1.0: USB Mass Storage device detected
      10. [31247.519245] scsi host11: usb-storage 1-1.4:1.0
      11. [31248.575384] scsi 11:0:0:0: Direct-Access Verbatim STORE N GO 5.00 PQ: 0 ANSI: 6\
      12. [31248.576714] sd 11:0:0:0: Attached scsi generic sg0 type 0
      13. [31249.372751] sd 11:0:0:0: [sda] 15646720 512-byte logical blocks: (8.01 GB/7.46 GiB)
      14. [31249.373725] sd 11:0:0:0: [sda] Write Protect is off
      15. [31249.373750] sd 11:0:0:0: [sda] Mode Sense: 23 00 00 00
      16. [31249.374709] sd 11:0:0:0: [sda] No Caching mode page found
      17. [31249.374729] sd 11:0:0:0: [sda] Assuming drive cache: write through
      18. [31249.399695] sda: sda1
      19. [31249.404068] sd 11:0:0:0: [sda] Attached SCSI removable disk
      20. [UPDATE 2016-08-21 to the following section for formatting and mounting the hard disk: Now we use gdisk instead of fdisk. See full information in the yellow section at the bottom]
      21. mark@redhen3rpi ~ $ sudo fdisk -l
      22. Device Boot Start End Blocks Id System
      23. /dev/sda1 2 15646719 7823359 b W95 FAT32
        1. mark@redhen3rpi ~ $ sudo fdisk /dev/sda
        2. Command (m for help): d
        3. Selected partition 1
        4. Command (m for help): n
        5. Partition type:
        6. p primary (0 primary, 0 extended, 4 free)
        7. e extended
        8. Select (default p): p
        9. Partition number (1-4, default 1):
        10. Using default value 1
        11. First sector (2048-15646719, default 2048):
        12. Using default value 2048
        13. Last sector, +sectors or +size{K,M,G} (2048-15646719, default 15646719):
        14. Using default value 15646719
        15. Command (m for help): w
        16. The partition table has been altered!
        17. Calling ioctl() to re-read partition table.
        18. Syncing disks.
        19. [Now let's check that it's actually connected and with its partition:]
        20. mark@redhen3rpi ~ $ sudo fdisk -l
      24. Disk /dev/sda: 8011 MB, 8011120640 bytes
      25. 8 heads, 40 sectors/track, 48896 cylinders, total 15646720 sectors
      26. Units = sectors of 1 * 512 = 512 bytes
      27. Sector size (logical/physical): 512 bytes / 512 bytes
      28. I/O size (minimum/optimal): 512 bytes / 512 bytes
      29. Disk identifier: 0x00000000
      30. Device Boot Start End Blocks Id System
      31. /dev/sda1 2048 15646719 7822336 83 Linux
      32. [Now let's format it for Linux as ext4:]
      33. mark@redhen3rpi ~ $ sudo mkfs -t ext4 /dev/sda1
      34. mke2fs 1.42.5 (29-Jul-2012)
      35. Filesystem label=
      36. OS type: Linux
      37. Block size=4096 (log=2)
      38. Fragment size=4096 (log=2)
      39. Stride=0 blocks, Stripe width=0 blocks
      40. 489600 inodes, 1955584 blocks
      41. 97779 blocks (5.00%) reserved for the super user
      42. First data block=0
      43. Maximum filesystem blocks=2004877312
      44. 60 block groups
      45. 32768 blocks per group, 32768 fragments per group
      46. 8160 inodes per group
      47. Superblock backups stored on blocks:
      48. 32768, 98304, 163840, 229376, 294912, 819200, 884736, 1605632
      49. Allocating group tables: done
      50. Writing inode tables: done
      51. Creating journal (32768 blocks): done
      52. Writing superblocks and filesystem accounting information: done
      53. [Now let's mount it as V8usbstick]
      54. mark@redhen3rpi ~ $ cd /mnt
      55. mark@redhen3rpi /mnt $ sudo mkdir V8usbstick
      56. [sudo] password for mark:
      57. mark@redhen3rpi /mnt $ sudo mount /dev/sda1 /mnt/V8usbstick
      58. mark@redhen3rpi /mnt $ cd V8usbstick
      59. [Now let's check that we can use V8usbstick]
      60. mark@redhen3rpi /mnt/V8usbstick $ sudo touch hello
      61. mark@redhen3rpi /mnt/V8usbstick $ ls
      62. hello lost+found
      63. [OK, as we see, that worked:]
      64. mark@redhen3rpi /mnt/V8usbstick $ df
      65. Filesystem 1K-blocks Used Available Use% Mounted on
      66. /dev/sda1 7568312 17196 7143616 1% /mnt/V8usbstick
      67. [OK, now let's unmount & eject it:]
      68. mark@redhen3rpi ~ $ sudo umount /dev/sda1
      69. mark@redhen3rpi ~ $ sudo eject /dev/sda
    3. Can we add a large external hard drive? Yes. Here is how we added a USB2 Western Digital 1TB external hard disk. It is importat that the WD hard disk have its own independent power supply: the RPi cannot power such a drive through its USB ports.
      1. First, connect the hard drive to a USB port on the RPi.
      2. mark@redhen3rpi ~ $ dmesg | tail -30
        1. [ 2397.017950] usb 1-1.5: new high-speed USB device number 5 using dwc_otg
        2. [ 2397.119128] usb 1-1.5: New USB device found, idVendor=1058, idProduct=1110
        3. [ 2397.119154] usb 1-1.5: New USB device strings: Mfr=1, Product=2, SerialNumber=3
        4. [ 2397.119172] usb 1-1.5: Product: My Book 1110
        5. [ 2397.119189] usb 1-1.5: Manufacturer: Western Digital
        6. [ 2397.119206] usb 1-1.5: SerialNumber: 5743415A4130323333333034
        7. [ 2397.120394] usb-storage 1-1.5:1.0: USB Mass Storage device detected
        8. [ 2397.122856] scsi host0: usb-storage 1-1.5:1.0
        9. [ 2398.120384] scsi 0:0:0:0: Direct-Access WD My Book 1110 2018 PQ: 0 ANSI: 4
        10. [ 2398.124489] scsi 0:0:0:1: CD-ROM WD Virtual CD 1110 2018 PQ: 0 ANSI: 4
        11. [ 2398.124976] sd 0:0:0:0: [sda] 1952151552 512-byte logical blocks: (999 GB/930 GiB)
        12. [ 2398.128851] sd 0:0:0:0: [sda] Write Protect is off
        13. [ 2398.128900] sd 0:0:0:0: [sda] Mode Sense: 2b 00 10 08
        14. [ 2398.132293] sd 0:0:0:0: [sda] No Caching mode page found
        15. [ 2398.132332] sd 0:0:0:0: [sda] Assuming drive cache: write through
        16. [ 2398.139498] scsi 0:0:0:2: Enclosure WD SES Device 2018 PQ: 0 ANSI: 4
        17. [ 2398.158885] sd 0:0:0:0: Attached scsi generic sg0 type 0
        18. [ 2398.159276] scsi 0:0:0:1: Attached scsi generic sg1 type 5
        19. [ 2398.164808] sda: sda1
        20. [ 2398.170516] scsi 0:0:0:2: Attached scsi generic sg2 type 13
        21. [ 2398.176030] sr 0:0:0:1: [sr0] scsi3-mmc drive: 51x/51x caddy
        22. [ 2398.176114] cdrom: Uniform CD-ROM driver Revision: 3.20
        23. [ 2398.178107] sd 0:0:0:0: [sda] Attached SCSI disk
        24. [ 2398.179302] sr 0:0:0:1: Attached scsi CD-ROM sr0
        25. [Note that this drive notoriously contains a very small partition with a virtual CD for "WD Smartware" that is impervious to reformatting. We'll just ignore it. For the future, better to have a clean hard drive formatted for Linux or Windows, and then to reformat it for raspbian, as below.]
        26. mark@redhen3rpi ~ $ sudo fdisk -l
        27. Disk /dev/sda: 999.5 GB, 999501594624 bytes
        28. 255 heads, 63 sectors/track, 121515 cylinders, total 1952151552 sectors
        29. Units = sectors of 1 * 512 = 512 bytes
        30. Sector size (logical/physical): 512 bytes / 512 bytes
        31. I/O size (minimum/optimal): 512 bytes / 512 bytes
        32. Disk identifier: 0x00000000
        33. Device Boot Start End Blocks Id System
        34. /dev/sda1 2 1952151551 976075775 b W95 FAT32
        35. mark@redhen3rpi ~ $ sudo fdisk /dev/sda
        36. Command (m for help): d
        37. Selected partition 1
        38. Command (m for help): n
        39. Partition type:
        40. p primary (0 primary, 0 extended, 4 free)
        41. e extended
        42. Select (default p): p
        43. Partition number (1-4, default 1):
        44. Using default value 1
        45. First sector (2048-1952151551, default 2048):
        46. Using default value 2048
        47. Last sector, +sectors or +size{K,M,G} (2048-1952151551, default 1952151551):
        48. Using default value 1952151551
        49. Command (m for help): w
        50. The partition table has been altered!
        51. Calling ioctl() to re-read partition table.
        52. Syncing disks.
        53. mark@redhen3rpi /dev $ sudo fdisk -l
        54. Disk /dev/sda: 999.5 GB, 999501594624 bytes
        55. 208 heads, 36 sectors/track, 260704 cylinders, total 1952151552 sectors
        56. Units = sectors of 1 * 512 = 512 bytes
        57. Sector size (logical/physical): 512 bytes / 512 bytes
        58. I/O size (minimum/optimal): 512 bytes / 512 bytes
        59. Disk identifier: 0x00000000
        60. Device Boot Start End Blocks Id System
        61. /dev/sda1 2048 1952151551 976074752 83 Linux
        62. mark@redhen3rpi ~ $ sudo mkfs -t ext4 /dev/sda1
        63. mke2fs 1.42.5 (29-Jul-2012)
        64. Filesystem label=
        65. OS type: Linux
        66. Block size=4096 (log=2)
        67. Fragment size=4096 (log=2)
        68. Stride=0 blocks, Stripe width=0 blocks
        69. 61005824 inodes, 244018688 blocks
        70. 12200934 blocks (5.00%) reserved for the super user
        71. First data block=0
        72. Maximum filesystem blocks=0
        73. 7447 block groups
        74. 32768 blocks per group, 32768 fragments per group
        75. 8192 inodes per group
        76. Superblock backups stored on blocks:
        77. 32768, 98304, 163840, 229376, 294912, 819200, 884736, 1605632, 2654208,
        78. 4096000, 7962624, 11239424, 20480000, 23887872, 71663616, 78675968,
        79. 102400000, 214990848
        80. Allocating group tables: done
        81. Writing inode tables: done
        82. Creating journal (32768 blocks): done
        83. Writing superblocks and filesystem accounting information: done
        84. mark@redhen3rpi ~ $ cd /mnt
        85. mark@redhen3rpi /mnt $ sudo mkdir HD1
        86. mark@redhen3rpi /mnt $ sudo mount /dev/sda1 /mnt/HD1
        87. mark@redhen3rpi /mnt $ cd HD1
        88. [Now let's check that we can use HD1]
        89. mark@redhen3rpi /mnt/HD1 $ sudo touch hello
        90. mark@redhen3rpi /mnt/HD1 $ ls
        91. hello lost+found
        92. [OK, as we see, that worked:]
        93. mark@redhen3rpi /mnt/HD1 $ df
        94. Filesystem 1K-blocks Used Available Use% Mounted on
        95. /dev/sda1 960628088 73364 911734604 1% /mnt/HD1
        96. We want all heavy work done on HD1 rather than on the microSD card. So make a symlink
        97. $ ln -s /mnt/HD1 HD1
        98. so that the directory HD1 shows up in the home directory of the user. Users should work on HD1 exclusively.
        99. For the future: consider the possibility of moving everything over to HD1 so as to avoid the corruption that can be induced by writing the microSD card: https://www.raspberrypi.org/forums/viewtopic.php?f=29&t=44177
      3. Moved the RPi to a university network. It now has a static IP address. Remounted the external hard drive HD1 with sudo mount /dev/sda1 /mnt/HD1.
      4. Now there are two Red Hen devices in one drawer in Crawford Hall, Case Western Reserve University:
          1. On the left, redhen1, a macmini, with a USB extender connecting to an Elgato EyeTV Hybrid off-screen. On the right, redhen3rpi, a Raspberry Pi 2 B, with a USB connection to its external hard disk in the upper right. redhen3rpi and an HDHomeRun (in the drawer below) are on the same network.
      1. [UPDATE 2015-06-20: Debian Jessie is now available for direct installation, so this step should be simpler.] Interlude: Francis Steen ran the script needed to upgrade Debian Wheezy to Debian Jessie, installed hdhomerun-config, uuid, gpac, ffmpeg, and established the usual tunnel (with RedHen scripts both /usr/local/bin/tunneld and backup /usr/local/bin/tunnel) between redhen3rpi and the rest of the Red Hen network. A few notes:
      2. $ sudo apt-get update
      3. $ sudo apt-get install wajig
      4. $ sudo apt-get upgrade
      5. $ sudo apt-get dist-upgrade
      6. $ sudo vi /etc/apt/sources.list
      7. Should have:
      8. deb http://mirrordirector.raspbian.org/raspbian/ jessie main contrib non-free rpi
      9. $ sudo apt-get upgrade
      10. $ sudo shutdown -r now
      11. $ sudo apt-get install ca-certificates
      12. $ sudo apt-get dist-upgrade
      13. $ sudo apt-get install git-core
      14. $ sudo apt-get install rpi-update
      15. $ sudo rpi-update
      16. $ sudo shutdown -r now
      17. $ sudo apt-get install eject
      18. $ sudo apt-get install uuid
    1. [UPDATE 2015-06-20. ffmpeg can be installed pre-compiled from deb-multimedia. The
    2. /etc/apt/sources.list file, which specifies sources for updates, now has the lines:
    3. # RPI Sources.list 2015-06-20
    4. # Stable official Debian
    5. deb http://mirrordirector.raspbian.org/raspbian/ jessie main contrib non-free rpi
    6. # Uncomment line below then 'apt-get update' to enable 'apt-get source'
    7. #deb-src http://archive.raspbian.org/raspbian/ wheezy main contrib non-free rpi
    8. # Debian jessie security team
    9. deb http://security.debian.org jessie/updates main
    10. # Debian Multimedia (http://www.deb-multimedia.org/)
    11. deb http://www.deb-multimedia.org jessie main
    12. #deb http://www.deb-multimedia.org jessie-backport main
    13. #deb http://www.deb-multimedia.org stretch main
    14. #deb http://www.deb-multimedia.org sid main]
    15. Is RPi powerful enough to compress a captured broadcast (.ts, .mpg) to .mp4? Yes. Francis Steen has deployed the method used at UCLA (two-pass ffmpeg) on redhen3rpi on a file captured on redhen1, with good results.
        1. Note for future: handbrake does not seem to be part of Debian Jessie. Installing handbrake on Debian Jessie: http://linuxg.net/how-to-install-handbrake-0-9-9-on-debian-jessie-debian-sid-and-debian-wheezy/ .
      1. Is RPi powerful enough to extract closed-captions? Yes. First, install ccextractor by getting the source:
      2. ==============
      3. csa@redhen3rpi ~/software $ lynx http://ccextractor.sourceforge.net/download-ccextractor.html
      4. csa@redhen3rpi ~/software $ ls
      5. ccextractor.src.0.77.zip
      6. csa@redhen3rpi ~/software $ ls -l
      7. total 1264
      8. -rw-r--r-- 1 csa csa 1292599 Jun 20 19:27 ccextractor.src.0.77.zip
      9. csa@redhen3rpi ~/software $ unzip *p
      10. Archive: ccextractor.src.0.77.zip
      11. creating: ccextractor.0.77/
      12. [etc. etc. etc.]
      13. csa@redhen3rpi ~/software $ ls -l
      14. total 1268
      15. drwxr-xr-x 9 csa csa 4096 Jun 20 18:03 ccextractor.0.77
      16. -rw-r--r-- 1 csa csa 1292599 Jun 20 19:27 ccextractor.src.0.77.zip
      17. csa@redhen3rpi ~/software $ cd *77
      18. csa@redhen3rpi ~/software/ccextractor.0.77 $ ls
      19. Dictionary docs linux mac OpenBSD src windows
      20. csa@redhen3rpi ~/software/ccextractor.0.77 $ cd linux
      21. csa@redhen3rpi ~/software/ccextractor.0.77/linux $ ls -l
      22. total 12
      23. -rwxr-xr-x 1 csa csa 245 Jun 20 14:29 build
      24. -rwxr-xr-x 1 csa csa 248 Jun 20 18:02 builddebug
      25. -rw-r--r-- 1 csa csa 3448 Jun 20 14:29 Makefile
      26. csa@redhen3rpi ~/software/ccextractor.0.77/linux $ ./build
      27. csa@redhen3rpi ~/software/ccextractor.0.77/linux $ ls -l
      28. total 1620
      29. -rwxr-xr-x 1 csa csa 245 Jun 20 14:29 build
      30. -rwxr-xr-x 1 csa csa 248 Jun 20 18:02 builddebug
      31. -rwxr-xr-x 1 csa csa 1642552 Jun 20 19:30 ccextractor
      32. -rw-r--r-- 1 csa csa 3448 Jun 20 14:29 Makefile
      33. csa@redhen3rpi ~/software/ccextractor.0.77/linux $ mv ccextractor ccextractor-0.77
      34. csa@redhen3rpi ~/software/ccextractor.0.77/linux $ sudo cp cc* /usr/local/bin
      35. csa@redhen3rpi ~/software/ccextractor.0.77/linux $ cd /usr/local/bin
      36. csa@redhen3rpi /usr/local/bin $ ls -l
      37. total 1620
      38. -rwxr-xr-x 1 root staff 1642552 Jun 20 19:31 ccextractor-0.77
      39. drwxr-sr-x 6 root staff 4096 Jun 19 18:45 hdhomerun_config_gui
      40. drwxrwsr-x 2 root indiecity 4096 May 6 19:17 indiecity
      41. drwxr-sr-x 2 root staff 4096 Jun 19 20:06 libhdhomerun
      42. csa@redhen3rpi /usr/local/bin $ sudo ln -s ccextractor-0.77 ccextractor
      43. ==============
      44. Steen tweaked and ran cc-extract-rpi on an mpeg file. It works, except for one minor remaining issue: the timestamps are inaccurate. Perhaps a setting in CCextractor needs to be changed.
      45. Can RPi capture a broadcast? Yes.
        1. Establish an HDHomeRun. That is, plug an HDHomeRun into the same internet subnet to which the RPi is connected. [UPDATE 2015-06-20: hdhomerun_config is present in the Debian repository, so can be installed directly instead of compiled from source.] From http://www.silicondust.com/support/downloads/ , follow the instructions for Linux Downloads, i.e., using a text browser like lynx (sudo apt-get install lynx), download libhdhomerun (source) and HDHomeRun Config GTK (source). These are .tgz files. Extract them with
        2. $ tar -xvzf hdhomerun_config_gui_20150615.tgz
        3. and
        4. $ tar -xvzf libhdhomerun_20150615.tgz
        5. From the hdhomerun_config_gui directory, run:
        6. $ sudo ./configure
        7. $ sudo make
        8. $ sudo make install
        9. Then go looking for the HDHomeRun on the subnet:
        10. $ ./hdhomerun_config discover
        11. And hope the response is something like: hdhomerun device 103B659E found at 129.22.115.124.
        12. Now you know the device number of your HDHomeRun (103B659E) and its IP address (129.22.115.124).
        13. Then find the model of your HDHomeRun:
        14. $ ./hdhomerun_config 103B659E get /sys/model
        15. And hope the response is something like: hdhomerun3_atsc
        16. Then find the firmware version you are currently running:
        17. $ ./hdhomerun_config 103B659E get /sys/version
        18. And hope the response is something like: 20130328
        19. Then find the latest firmware for your model on the silicondust.com site and download it. It will have a name like hdhomerun3_atsc_firmware_20150604.bin once you download it.
        20. Then upgrade your firmware:
        21. $ ./hdhomerun_config 103B659E upgrade ~/HD1/hdhomerun3_atsc_firmware_20150604.bin
        22. and hope you see something like:
        23. uploading firmware...upgrading firmware...rebooting...upgrade complete - now running firmware 20150604
      46. Now configure the HDHomeRun using hdhomerun_config. There are 2 tuners in the HDHomeRun on redhen3rpi, numbered 0 and 1.
          1. Set a variable for DEVICE and a variable for TUNER
          2. $ DEVICE=103B659E TUNER=0
          3. Get or set the channelmap:
          4. $ ./hdhomerun_config $DEVICE get /tuner0/channelmap
          5. And hope the response is what you want to see, e.g.: us-cable
          6. Scan tuner 0 for channels:
            1. $ sudo ./hdhomerun_config $DEVICE scan /tuner$TUNER scan0us-cable.log
            2. And hope it finds some, e.g.:
            3. SCANNING: 69000000 (us-irc:4, us-cable:4)
            4. LOCK: qam256 (ss=93 snq=69 seq=100)
            5. PROGRAM 1: 4.1 WQHS
            6. PROGRAM 2: 4.2 WMFD
            7. PROGRAM 3: 4.3 CNN
            8. PROGRAM 4: 4.4 CNN HLN
            9. PROGRAM 5: 4.5 FOX New
            10. PROGRAM 6: 4.6 MSNBC
            11. PROGRAM 7: 4.7 CNBC
            12. PROGRAM 8: 4.8 CSPAN
          7. So, let's tune to us-cable channel 4:
          8. $ sudo ./hdhomerun_config $DEVICE set /tuner$TUNER/channel auto:4
          9. And check how we are doing:
          10. $ sudo ./hdhomerun_config $DEVICE get /tuner$TUNER/status
          11. We get the response: ch=auto:4 lock=qam256 ss=94 snq=0 seq=0 bps=38813728 pps=0
          12. Let's select program 3 (4.3); that's CNN, as we learned above:
          13. $ ./hdhomerun_config $DEVICE set /tuner$TUNER/program 3
          14. Now, let's start capturing CNN for a while
          15. $ ./hdhomerun_config $DEVICE save /tuner$TUNER ~/HD1/TestCapture2015-06-19.ts
          16. And we see (note that we stop the recording with control-C):
          17. .................................................n....t........^C
          18. -- Video statistics --
          19. 233303 packets received, 0 overflow errors, 1 network errors, 6 transport errors, 0 sequence errors
          20. Similar tests show that we can capture MSNBC, Fox, etc.
          21. And when we move the captured file to a Mac and look at it in VLC, we see the subtitles:
Fox News with subtitles
Update 2016-08-21: Using gdisk instead of fdisk:
csa@rusalka:~ $ sudo just install gdisk
Reading package lists... Done
Building dependency tree       
Reading state information... Done
The following NEW packages will be installed:
  gdisk
0 upgraded, 1 newly installed, 0 to remove and 0 not upgraded.
Need to get 193 kB of archives.
After this operation, 799 kB of additional disk space will be used.
Get:1 http://mirror.ox.ac.uk/sites/archive.raspbian.org/archive/raspbian/ jessie/main gdisk armhf 0.8.10-2 [193 kB]
Fetched 193 kB in 0s (317 kB/s)
Selecting previously unselected package gdisk.
(Reading database ... 89584 files and directories currently installed.)
Preparing to unpack .../gdisk_0.8.10-2_armhf.deb ...
Unpacking gdisk (0.8.10-2) ...
Processing triggers for man-db (2.7.0.2-5) ...
Setting up gdisk (0.8.10-2) ...
csa@rusalka:~ $ sudo gdisk /dev/sda
GPT fdisk (gdisk) version 0.8.10
Partition table scan:
  MBR: MBR only
  BSD: not present
  APM: not present
  GPT: not present
***************************************************************
Found invalid GPT and valid MBR; converting MBR to GPT format
in memory. THIS OPERATION IS POTENTIALLY DESTRUCTIVE! Exit by
typing 'q' if you don't want to convert your MBR partitions
to GPT format!
***************************************************************

Warning! Secondary partition table overlaps the last partition by

33 blocks!
You will need to delete this partition or resize it in another utility.
Command (? for help): ?
b       back up GPT data to a file
c       change a partition's name
d       delete a partition
i       show detailed information on a partition
l       list known partition types
n       add a new partition
o       create a new empty GUID partition table (GPT)
p       print the partition table
q       quit without saving changes
r       recovery and transformation options (experts only)
s       sort partitions
t       change a partition's type code
v       verify disk
w       write table to disk and exit
x       extra functionality (experts only)
?       print this menu
Command (? for help): x
Expert command (? for help): ?
a       set attributes
c       change partition GUID
d       display the sector alignment value
e       relocate backup data structures to the end of the disk
g       change disk GUID
h       recompute CHS values in protective/hybrid MBR
i       show detailed information on a partition
l       set the sector alignment value
m       return to main menu
n       create a new protective MBR
o       print protective MBR data
p       print the partition table
q       quit without saving changes
r       recovery and transformation options (experts only)
s       resize partition table
t       transpose two partition table entries
u       replicate partition table on new device
v       verify disk
w       write table to disk and exit
z       zap (destroy) GPT data structures and exit
?       print this menu
Expert command (? for help): z
About to wipe out GPT on /dev/sda. Proceed? (Y/N): y
GPT data structures destroyed! You may now partition the disk using fdisk or
other utilities.
Blank out MBR? (Y/N): y
csa@rusalka:~ $ sudo gdisk /dev/sda
GPT fdisk (gdisk) version 0.8.10
Partition table scan:
  MBR: not present
  BSD: not present
  APM: not present
  GPT: not present
Creating new GPT entries.
Command (? for help): n
Partition number (1-128, default 1): 
First sector (34-3906963422, default = 2048) or {+-}size{KMGTP}: 
Last sector (2048-3906963422, default = 3906963422) or {+-}size{KMGTP}: 
Current type is 'Linux filesystem'
Hex code or GUID (L to show codes, Enter = 8300): 
Changed type of partition to 'Linux filesystem'
Command (? for help): i
Using 1
Partition GUID code: 0FC63DAF-8483-4772-8E79-3D69D8477DE4 (Linux filesystem)
Partition unique GUID: 7C1DEEB3-DFDE-4502-ADFE-1DF9323DADCE
First sector: 2048 (at 1024.0 KiB)
Last sector: 3906963422 (at 1.8 TiB)
Partition size: 3906961375 sectors (1.8 TiB)
Attribute flags: 0000000000000000
Partition name: 'Linux filesystem'
Command (? for help): w
Final checks complete. About to write GPT data. THIS WILL OVERWRITE EXISTING
PARTITIONS!!

Do you want to proceed? (Y/N): y

OK; writing new GUID partition table (GPT) to /dev/sda.
The operation has completed successfully.
csa@rusalka:~ $ mkfs.xfs /dev/sda1
bash: mkfs.xfs: command not found
csa@rusalka:~ $ sudo just install xfsprogs
Reading package lists... Done
Building dependency tree       
Reading state information... Done
The following extra packages will be installed:
  libreadline5
Suggested packages:
  xfsdump attr quota
The following NEW packages will be installed:
  libreadline5 xfsprogs
0 upgraded, 2 newly installed, 0 to remove and 0 not upgraded.
Need to get 746 kB of archives.
After this operation, 3,453 kB of additional disk space will be used.
Do you want to continue? [Y/n] 
Get:1 http://mirror.ox.ac.uk/sites/archive.raspbian.org/archive/raspbian/ jessie/main libreadline5 armhf 5.2+dfsg-2 [131 kB]
Get:2 http://mirror.ox.ac.uk/sites/archive.raspbian.org/archive/raspbian/ jessie/main xfsprogs armhf 3.2.1 [614 kB]
Fetched 746 kB in 1s (746 kB/s)
Selecting previously unselected package libreadline5:armhf.
(Reading database ... 89605 files and directories currently installed.)
Preparing to unpack .../libreadline5_5.2+dfsg-2_armhf.deb ...
Unpacking libreadline5:armhf (5.2+dfsg-2) ...
Selecting previously unselected package xfsprogs.
Preparing to unpack .../xfsprogs_3.2.1_armhf.deb ...
Unpacking xfsprogs (3.2.1) ...
Processing triggers for man-db (2.7.0.2-5) ...
Setting up libreadline5:armhf (5.2+dfsg-2) ...
Setting up xfsprogs (3.2.1) ...
Processing triggers for libc-bin (2.19-18+deb8u4) ...
csa@rusalka:~ $ sudo mkfs.xfs /dev/sda1
mkfs.xfs: /dev/sda1 appears to contain an existing filesystem (ntfs).
mkfs.xfs: Use the -f option to force overwrite.
csa@rusalka:~ $ sudo mkfs.xfs /dev/sda1 -f
meta-data=/dev/sda1              isize=256    agcount=4, agsize=122092543 blks
         =                       sectsz=512   attr=2, projid32bit=1
         =                       crc=0        finobt=0
data     =                       bsize=4096   blocks=488370171, imaxpct=5
         =                       sunit=0      swidth=0 blks
naming   =version 2              bsize=4096   ascii-ci=0 ftype=0
log      =internal log           bsize=4096   blocks=238461, version=2
         =                       sectsz=512   sunit=0 blks, lazy-count=1
realtime =none                   extsz=4096   blocks=0, rtextents=0
csa@rusalka:~ $ mount /mnt/HD1
csa@rusalka:~ $ df -h
Filesystem      Size  Used Avail Use% Mounted on
/dev/root        29G  2.5G   25G   9% /
devtmpfs        459M     0  459M   0% /dev
tmpfs           463M     0  463M   0% /dev/shm
tmpfs           463M  6.4M  457M   2% /run
tmpfs           5.0M  4.0K  5.0M   1% /run/lock
tmpfs           463M     0  463M   0% /sys/fs/cgroup
/dev/mmcblk0p1   63M   21M   43M  33% /boot
tmpfs            93M     0   93M   0% /run/user/1000
tmpfs            93M     0   93M   0% /run/user/1001
/dev/sda1       1.9T   33M  1.9T   1% /mnt/HD