Mails from Francis - Chapter 1

introduction

Let's start with some rules (working name: Francis's Noble Sixfold Path)

Assess the costs and benefits of possible enhancements, and prioritize them carefully -- your time is limited
Implement the changes so that they can easily and reliably be undone and the system restored to its prior known working state -- and document how it! (e.g., changelogs)
Don't assume you foresee the consequences of your changes -- always test thoroughly first!
Postpone changes until you and the system find the time to perform all necessary tests
Monitor the operation of the system after you make a change and verify everything is working correctly
Back up your changes

The joy of building Red Hen is that the systems we build actually work -- on their own, unattended, while we do other things. Building them so that this is true takes careful discipline.

Possible additions to Francis's Red Hen Noble Sixfold Path being considered at the moment are:

7. Remember, there's a difference between knowing the Red Hen Noble Sixfold Path and walking the Red Hen Noble Sixfold Path.

8. May the Force be with you.

If we add them, we'll have an eightfold path. Just need two more and we'll have Francis's Red Hen 10 Commandments.

On a more serious note- all the six points above, like the aviation regulations, were written in blood- of the innocent files I deleted by accident or the important system files I hacked beyond recognition, without renaming or backupping once.

Oh, this is going to be a loooong section. Loads of useful information. Hundreds of mails, hours, days, weeks of valuable time of an eminent UCLA professor spent on helping a capture station captain (the rank I promoted myself to) in a remote corner of Poland.

I don't know where to start so I'll try chronologically and then perhaps try some thematic ordering.

I'm in Athens, just visited the Acropolis [...]; leaving for Rome tomorrow. - this is where Francis did the most initial work on Polish capture station.

It's great we're getting a signal -- and both Polish and Czech! I configured the hard drive -- wiped the MS partitions and created a new gpt partition with gdisk, rebooted, formatted the partition with xfs (mkfs.xfs /dev/sda1), and mounted as /mnt/HD1. I added symlinks to the new drive in /home/csa, /, /nest/cfg, /mnt, and /nest/tuners, using rusalka as a model - so just imagine, please, Francis on the top of Acropolis, wiping the MS partitions and adding symlinks so that the next little capture station can run.

freeguide

The next step is to run Freeguide in VNC and configure for Poland. Not sure yet how we generate two configurations (add for Czechia). First let's confirm we can download the Polish schedule (modify xmltv-download once we have the Freeguide configuration).- most of it gibberish at the time for me. But now I can explain- VNC is the program for connecting the little Raspberry PI computer to your laptop. When you make it work, you can enjoy the graphical user interface, GUI, on RPI- an abbreviation for Rasperry PI computer that fits into a matchbox (almost). Freeguide is a program that helps to schedule the automated recording of tv programs but it doesn't work for Poland or Czechia (Francis and Mark did't know it at the time).

putty

Once you have the key converted and installed in puTTY, you should be able to connect. If you'd like to help out, the next task is to generate the correct values for /nest/tuners/lineup, using the information from the scan for the networks we're interested in.P.S. I placed a lineup template and the scan file in your home directory on dola. We still need to find the source of the xmltv program information. - puTTY: a free ssh telnet client, says Google. Oh yes, thank you Google - die soon and good riddance to you, one would like to add. Let me explain what puTTY is. When you install it and make it run (not difficult, one of the things you need for it is the IP address of the RPI, see 'the router' section) you get he Matrix black box (like the DOS command prompt but oh so much cooler). And you'll be able to do ANYTHING on the RPI. Full control. You just have to know which command to use or which script to run ( a command is also a sort of script, so this line is like redundant and so is like and what would life be without a little redundancy now and then, eh?). So, run putty, it'll open the box, and enter

And of course you will get 'bash: the command not found". Not found? Oh, yes? Then why don't you make it you business to go look for it? Anyway, back to the mails.

generate the correct values for /nest/tuners/lineup, using the information from the scan for the networks we're interested in - generate the correct values? Who does he (Francis) think I am? A correct value generator? and 'using the information from the scan'? Which scan? What's going on? Where's the PANIC button?

navigating the files

And what the hell is /nest/tuners/lineup? Well, now- older wiser but still a dummy - I can explain: /nest/tuners/ is a DIRECTORY (like a folder in Windows- there's a folder NEST and it contains a folder TUNERS). If you want to know which folder you are in now, in putty - write pwd (print working directory) and press Enter.

aha - so I am in folder csa which is a subfolder of home. So far so good. Now, the next important thing is to know what files or directories (folders) there are. and for that - write ls (like list) and press Enter.

xmltv, for example, is a directory. To go there, write cd xmltv and press Enter. To go back again go cd .. and Enter. And so on. It's all in Google. Not rocket science, really.

lineup

So we're still here: generate the correct values for /nest/tuners/lineup, using the information from the scan for the networks we're interested in - let's find it. We already know how to go to /nest/tuners directory (cd /nest/tuners) but instead we may request the console (our Raspberry through it) to show it to us wherever we are. There are two ways of doing it:

1. SHOUT (but really loudly) - hey, you, Raspberry- show me the lineup now! (always works, please try now)

2. or write cat /nest/tuners/lineup and press Enter (if you don't want to wake the neighbours); cat is the command for showing the contents of a file. And we get (on my RPI) something like this:

So now at least we know that lineup is a file and it contains lines with some info about tv channels. OK. This is good. But we're still here (we'll never get out of this line):

generate the correct values for /nest/tuners/lineup, using the information from the scan for the networks we're interested in - this particular lineup is ready but it wasn't and that's why Francis talks about 'generating' (sounds like something from Dr Who - he meant 'finding'). So where to find those 'values for the lineup'? One way is to visit this site here but be warned- it's a scary place, don't go there at night. However, there is a simpler way. If you install the hdhomerun (I refuse from now to switch lower-upper case mid word for no reason) software for windows you will not only be able to watch tv on your laptop but you may do the scan there. Unfortunately, not all information in the lineup is to be found in the scan. It's a bit complicated - just mail me and we'll sort it out (jacekifa@gmail.com).

finding the recordings

We need to go to the next mail. For navigation in the file system, use the custom command "day" to move in the tv tree: day -- takes you to today's files

day 2013-05-03 ; day 5 (for five days ago) ; day - 1 (for one day earlier -- spaces on both sides of the minus sign) ; day + 4 (for four days later)

When you write day, for example, and press Enter it takes you to the catalog (folder) where today's recordings are and those catalogs in capture station sailors brogue (Francis and Mark speak like that all the time) is called the tv tree. and when you are in this catalog you can write ls to see what's in there:

video logs (1)

And if you want to see one of those files, use cat, for exampe: cat 2016-11-10_1600_PL_TVP1_Teleexpress-HD.log. It would take 5 minutes to write - such a long file name but there is a neat trick to it: mark the file name with the mouse (don't right click yet- just mark it) then write cat, press Space and then right click and Enter. Marking text in the black box (putty console, ssh, shut-up, telnet client, whatever) is a very helpful thing - whenever you mark anything you can then right-click and you will have it wherever your cursor is or you can even use it outside the console because when you mark, it puts the marked text in Windows (Mac OS, etc) clipboard. Anyway, when we do cat 2016-11-10_1600_PL_TVP1_Teleexpress-HD.log, we get:

I should explain that Teleexpress is the name of a news show my capture station is recording daily and the *.log file tells you how many errors there were. 67 today - not perfect but OK compared to 391032 'packets received'. And also - Francis and Mark wrote scripts that repair some of those errors, so all is well in dola capture station today (my Polish-Czech capture station is called dola but in fact I have two RPIs here, the other is called vila, I'll explain later).

test recording

Test recordings (issue commands from user csa only, see channel -h for a help screen): $ channel 1, 2min, TEST, 1, 888, "Test recording"

I never did at the time- things were happening fast, I wasn't sure of anything, mails were flying, Francis suggesting impossible things like 'generating vales for lineup', etc. I really didn't know what was happening - who are those people, Francis and Mark, I was asking myself, why do they know so much about bash and telnet clients and unix, linux, programming, all the technical stuff. Are they, just a possibility, some computer hackers, or contract killers^*, only posing as respectable professors? Well, I still have not quite ruled this out but, anyway, back to the subject.

Let's just do what Francis was suggesting now - just mark (right click/copy) from his mail: channel 1, 2min, TEST, 1, 888, "Test recording" and then go into the black box, right click and Enter.

I changed 2min to 1min not to wait too long. This is what we get:

A lot of chatter on the screen but WE DID IT! A 1 min test recording from channel 1. And channel 1 is defined in lineup. In my case it is Polish TV Channel 1. It has recorded. And how to find today's recordings? Have you been reading this text carefully? Of course we know it- just write day and Enter (in fact we don't need to because we are still in the same directory). And when we ls we get this now:

There are several additions to the catalog, because while I was typing, dola was busy recording Udalosti (Czech news) and Wiadomości (another Polish news show). But on the list above we have 2016-11-10_1856_PL_TVP1_TEST.mpg (purple colour- or color if ya are from Louisiana). And we did it with channel 1... etc. command. So this is what channel command does - it makes the RPI record things and it does it through hdhomerun device which converts the antenna signal into the digital signal and then, your RPI records it on the hard disc which is also connected to the router. And from the hard disc the files will be piped to California (where the palms, swimming pools, Porsche cars, Hollywood, and Francis, are). Well, truth be told, our test recording will never make it to California (because it has TEST in the name and will be ignored by the transfer script) but all the other recordings will. And from now on they will live a happy life in UCLA Newsscape Library, where hundreds of busy researchers will ponder over them, write articles about them, include them in their PhD theses, etc. I can only hope I can pipe myself there one day to see all those Porsches and swim in the swimming pools, etc. Let's see what's next.

crontab

the skill to learn is cron, the unix task scheduler. See the content of crontab -l - cron, eh? The skill to learn is cron. And this cron is the unix task scheduler, eh? You might as well have told me xmzqwerty!@#%^$*&xyz!?? and it would have meant exactly the same. But now I now have a vague idea. So crontab is a file in which you put commands that are to be executed reguraly, say daily recordings of tv programs. When we issue crontab -l (Enter), we get a lot of output on the screen, I will show you only some fragments:

So, the line:

31 07 * * * schedule TVP1 "Teleexpress" 1 777 "National afternoon news"

means that every day at exactly 7.31 a command will be executed. And this command has the format of schedule TVP1 "Teleexpress" 1 777 "National afternoon news" . And the line:

00 17 10 11 4 channel 1 15min "Teleexpress" 1 777 "National afternoon news"

means that at 17.00 on the 10th of November, 4th day of the week (Thursday) another command will be executed (channel 1 15min "Teleexpress" 1 777 "National afternoon news" ). But we already know what the channel command does - it records programs. We don't know yet what the schedule command does but more about it later.

If you wish to continue chronologically (read e-mails from Francis from cover to cover- recommended) go to Chapter 2.

___________

* - there's new evidence, however, that Mark and Francis might also be robots or aliens from outer space. This is what I found, for example, in one of Francis's mails (Nov. 2016): "What is the output for humans and how to you store intermediate output for further processing? What is the workflow from raw data to inferences?"

Page updated

Google Sites

Report abuse