Automatic scheduling

Task

Can you help establish automatic scheduling via crontab drawing on the Electronic Programming Guide (EPG) that usually accompanies any digital broadcast?

If so, write to

and we will connect you with a mentor.

Status

Provisionally solved 29 November 2015 by Steen and Fonseca. Pilot bash script running on Fonseca's Portugal capture station; python version under development. Longer-term solution is likely TVHeadend.

Schedule data sources

In the US, Schedules Direct provides television schedules for $25 a year; Red Hen subscribes and downloads the schedules daily via xmltv configured through FreeGuide.

In the EU, schedules are often available for free; this is for instance the case in Portugal. The grabbers are listed in the Raspbian software package xmltv-utils:

just list-files xmltv-util

For instance, there is /usr/bin/tv_grab_it. First run tv_grab_it --configure to choose which channels you want to download. Then running tv_grab_it with no arguments will output listings in XML format; we use the Red Hen script xmltv-download for this.

TVHeadend

Pete Broadwell and Peter Uhrig are experimenting with TVHeadend, which provides an online interface to an automatically updated schedule. For HDHomeRun devices, there is a native Linux driver, dvbhdhomerun, that works well with TVHeadend; both are installed on durga, but they are not in production.

Bash minimalism

Steen and Fonseca have implemented a hack on the Portugal capture station -- configure xmltv through FreeGuide and download the schedule daily, then parse it in the bash shell and generate the appropriate crontabs that schedule the recordings. This works for simple schedules and is fully automated.

FreeGuide

José configured FreeGuide for our four channels; here it is on fenix, through X11:

Here's the show we record -- and in the bottom right you see the download in progress:

We can get episode numbers for the telenovela -- I'd like to download the xml and keep it, as we do on cartago:

Search function for scheduling:

So it turns out Telejornal has a variable length -- how do we stay on top of this?

Schedule automation

Commercial software for recording television typically integrates the Electronic Program Guide, or EPG, to schedule the recordings. In the open source world we have xmltv, which provides television schedules in an xml format. The service is free in many countries, or available at a nominal cost. On many of its capture stations, Red Hen uses the unix scheduler crontab. For a full decade we have dreamed of connecting this system to the EPG. The simple task of the Portugal capture station and Fonseca's expert configuration of FreeGuide inspired us to take the plunge, and Red Hen now has a simple way to update recording schedules automatically, based on the downloaded xmltv broadcast schedule. The pilot version is operational on our new recording station in Portugal, in the form of a short bash script called schedule. A more robust version in python is under development; both are documented here.

Schedule automation lowers the cost of maintaining a recording station and makes it easier to maintain remotely. Commercial software may stall while insisting on an upgrade, or silently fail if an EPG subscription lapses. On the other hand, manually checking the recording schedule invites mistakes and omissions. A unix-based system that updates its own schedule based on xmltv downloads turned out to be surprisingly easy to implement and makes our capture system significantly more accurate and robust against human failure.

Bash script

The first-iteration schedule bash script simply greps the xmltv download for a named show, picks out the start and end times, and puts them into crontab. Here's the help screen:

$ schedule -h

Update the recording schedule in crontab using the downloaded schedule from xmltv.

Syntax (use s to simulate results, without changing the crontab):

schedule <network> <show name> [<comment>] [<tuner>] [s]

Enclose any element with a space in quotes:

schedule "RTP 1" "Telejornal" "National evening news" 4 s

The comment field and tuner number are optional.

The script is designed to run from crontab but can be run manually.

If you run this script for a show, updating that show will be added to crontab.

Here's the user output:

$ schedule RTP-1 Telejornal "National evening news" 2 s

Fixing to update the recording schedule for "Telejornal" on RTP-1 ...

34 07 * * * schedule RTP-1 "Telejornal" "National evening news" 2

00 20 28 11 * channel 1, 45, "Telejornal", 2, "National evening news"

00 20 29 11 * channel 1, 63, "Telejornal", 2, "National evening news"

00 20 30 11 * channel 1, 68, "Telejornal", 2, "National evening news"

00 20 01 12 * channel 1, 75, "Telejornal", 2, "National evening news"

00 20 02 12 * channel 1, 75, "Telejornal", 2, "National evening news"

00 20 03 12 * channel 1, 45, "Telejornal", 2, "National evening news"

00 20 04 12 * channel 1, 60, "Telejornal", 2, "National evening news"

The Telejornal schedule on RTP-1 has been updated at Sat Nov 28 17:28:06 UTC 2015 and will update itself daily.

This removes the otherwise tedious task of tracking the variations in show length you see above, from 45 minutes to 75.

The script has the nifty little feature that if you use it to schedule a show, it will by default write that scheduling request into crontab along with the rest of the schedule, which means it will automatically update itself. So you program the schedule once, and it keeps itself up to date.

Python script

The downloaded xmltv files are best parsed with python-xmltv, a dedicated parser. It's available in Debian Linux and is installed on the capture machine durga at UCLA.