Configure a new station

introduction

To install the Red Hen software on a new RPI capture station:

• Become root

• apt install wajig (for package management)

• cd /usr/bin ; ln -s wajig just (creates an alias)

Expand the root partition to fill the SD card:

• raspi-config --expand-rootfs

To define the locales (and fix the warning "-bash: warning: setlocale: LC_ALL: cannot change locale (en_US.UTF-8)", issue

• just reconfigure locales

Uncheck

[*] en_GB.UTF-8 UTF-8

and check these to keep the environment invariant across capture stations:

[*] en_US.UTF-8

[*] en_US ISO-8859-1

[*] en_US.ISO-8859-15

In addition, add these two for the relevant language:

[*] es_ES.UTF-8

[*] es_ES@euro ISO-8859-15

We may not need the last, but just in case.

Create /nest as user root and change permissions to csa:csa:

• mkdir /nest

• chown csa:csa /nest

Copying nest

The nest directory is copied in from cartago using the most recent source; it contains the relevant Red Hen software:

• csa@cartago:~$ rsync /home/dola/system/dola/nest/ csa@154.64.199.123:/nest/ -av (not the real IP)

• root@miocid: ~ # chown -R csa:csa /nest/ (not normally needed)

Install.sh

Before running install.sh, it's a good idea to set up the hard drive at /mnt/HD1, owned by csa:csa. If you do, install.sh will create the needed directories and symlinks. For instructions on formatting the hard drive, see the section below.

Modify /nest/cfg/redhen.config as appropriate, for example:

# Red Hen configuration file

location=Murcia, Spain

hostname=miocid.es

timezone=Europe/Madrid

e-mail=csa

essid="mt-CòllegeDeFrance"

psk="LeBlending"

The last two lines define wireless access. are for the wireless -- if Eduroam is available, see

• https://www.raspberrypi.org/forums/viewtopic.php?f=28&t=86253

• https://servicedesk.rose-hulman.edu/link/portal/710/4769/Article/372/How-do-I-connect-a-Raspberry-Pi-to-the-Eduroam-Wireless-Network

Then run install.sh as user root:

• root@miocid: ~ # ./install.sh (sparsely interactive -- press y or n as needed)

On a fresh installation, the script can take 2.5 hours to complete, mainly installing Debian packages.

Mail configuration

The mail configuration is for the exim4 mail transport agent (MTA). If it's possible to send mail out, we need the name of the institution's SMTP host. It could be very useful to allow us to generate e-mail notifications. Chose smarthost.

To rerun the MTA definition if needed,

just reconfigure exim4-config

See also the notes on timezone configuration that follows this information in Portugal capture station.

Find the tuners

• 1. cd /nest/tuners

  2. find-tuners.sh

  3. cp signal current

  4. /tv needs to be a symlink to /mnt/HD1/tv

Go ahead and create /mnt/HD1/spool and then symlink

ln -s /mnt/HD1/spool /mnt/spool

You'll need to be root and then change the permissions to csa:csa.

Appendix

Formatting the external hard drive hard drive

We typically use an externa USB hard drive for data storage. We might also use a USB stick; cf. formatting usb stick.

First, find it and unmount the hard drive, as root:

root@miocid:~ # df -h | grep sd

Filesystem Size Used Avail Use% Mounted on

/dev/sda2 4.6T 251M 4.6T 1% /media/pi/Seagate Expansion Drive

Here it's mounted itself as "/media/pi/Seagate Expansion Drive". Unmount it:

# umount /mnt/sda2

If you're curious, query it:

# fdisk -l /dev/sda

Device Start End Sectors Size Type

/dev/sda1 34 262177 262144 128M Microsoft reserved

/dev/sda2 264192 9767540735 9767276544 4.6T Microsoft basic data

Partition 2 does not start on physical sector boundary.

These partitions should be removed. We first zap everything (note we operate directly on /dev/sda, not on /dev/sda1 or 2):

root@miocid:~ # gdisk /dev/sda

GPT fdisk (gdisk) version 0.8.10

Command (? for help): x

Expert command (? for help): z

Blank out MBR? (Y/N): y

Then we create a single new partition of the GPT type:

root@miocid:~ # gdisk /dev/sda

GPT fdisk (gdisk) version 0.8.10

Creating new GPT entries.

Command (? for help): n

Partition number (1-128, default 1):

First sector (34-9767541133, default = 2048) or {+-}size{KMGTP}:

Last sector (2048-9767541133, default = 9767541133) or {+-}size{KMGTP}:

Current type is 'Linux filesystem'

Hex code or GUID (L to show codes, Enter = 8300):

Changed type of partition to 'Linux filesystem'

Command (? for help): i

Using 1

Partition GUID code: 0FC63DAF-8483-4772-8E79-3D69D8477DE4 (Linux filesystem)

Partition unique GUID: 5345F871-0A8D-42D2-ABB4-76D1CF1E6704

First sector: 2048 (at 1024.0 KiB)

Last sector: 9767541133 (at 4.5 TiB)

Partition size: 9767539086 sectors (4.5 TiB)

Attribute flags: 0000000000000000

Partition name: 'Linux filesystem'

Command (? for help): w

Final checks complete. About to write GPT data. THIS WILL OVERWRITE EXISTING

PARTITIONS!!

Do you want to proceed? (Y/N): y

OK; writing new GUID partition table (GPT) to /dev/sda.

The operation has completed successfully.

Next, we add a file system. For large drives that will receive large files, we use xfs:

root@miocid:~ # mkfs.xfs /dev/sda1 (apt-get install xfsprogs if need be, JW)

meta-data=/dev/sda1 isize=256 agcount=5, agsize=268435455 blks

= sectsz=4096 attr=2, projid32bit=1

= crc=0 finobt=0

data = bsize=4096 blocks=1220942385, imaxpct=5

= sunit=0 swidth=0 blks

naming =version 2 bsize=4096 ascii-ci=0 ftype=0

log =internal log bsize=4096 blocks=521728, version=2

= sectsz=4096 sunit=1 blks, lazy-count=1

realtime =none extsz=4096 blocks=0, rtextents=0

Verify:

root@miocid:~ # fdisk -l /dev/sda

Disk /dev/sda: 4.6 TiB, 5000981077504 bytes, 9767541167 sectors

Units: sectors of 1 * 512 = 512 bytes

Sector size (logical/physical): 512 bytes / 4096 bytes

I/O size (minimum/optimal): 4096 bytes / 4096 bytes

Disklabel type: gpt

Disk identifier: C6989A95-4C00-4B27-9BAA-AB24FB0E0C97

Device Start End Sectors Size Type

/dev/sda1 2048 9767541133 9767539086 4.6T Linux filesystem

Add a label -- note we now label the partition, /dev/sda1, NOT the drive, /dev/sda:

root@miocid:~ # xfs_admin -L HD1 /dev/sda1

writing all SBs

new label = "HD1"

Add the drive to the file system table -- nano /etc/fstab:

LABEL=HD1 /mnt/HD1 xfs auto,nofail,rw,user,exec,relatime 0 0

Create the mount point and give it to csa:

root@miocid:~ # md /mnt/HD1

root@miocid:~ # chown -R csa:csa /mnt/HD1

In a different screen window, mount the drive as user csa:

csa@miocid:~ $ mount /mnt/HD1

If you get this error (I did):

mount: unknown filesystem type 'xfs'

It's because install.sh refreshed the packages and installed a new kernel, after which a reboot is required. So reboot; the drive will now mount. Give it again to csa (important):

root@miocid:~ # chown -R csa:csa /mnt/HD1

Verify with df. Highly efficient people (tm) enlist the external hard drive prior to running install.sh; that way install.sh creates the needed directories on the new drive. I've never done this myself but it's clearly a good idea.

Now we do it manually -- complete the symlinks in /home/csa and /nest/tuners:

lrwxrwxrwx 1 15 Dec 3 09:25 mail -> HD1/system/mail

lrwxrwxrwx 1 10 Dec 3 09:25 system -> HD1/system

lrwxrwxrwx 1 15 Dec 3 09:25 todo -> HD1/system/todo

lrwxrwxrwx 1 6 Dec 3 09:25 ts -> HD1/ts

lrwxrwxrwx 1 6 Dec 3 09:25 tv -> HD1/tv

So issue

md -p HD1/system/mail HD1/system/todo HD1/ts HD1/tv /mnt/HD1/system/tuners/logs

Verify with the alias

ll

Getting things from backup on cartago

You can get scripts straight from the backups on cartago --

./yinfo-backin dola check-cc-single-05.sh

fail2ban

This is used for protecting the capture station from attacks; see Configure fail2ban.

Upgrading

Upgrading the operating system should be done by installing a new image; the normal Debian dist-upgrade procedure is not recommended for Raspberry Pi. However, we have in the past done this successfully as follows -- again, this is not recommended; check Raspberry Pi OS instructions.

Debian released Stretch in the summer of 2017, and Raspbian soon followed. We upgraded from Jessie to Stretch in October 2017.

1. Become root

csa@redhen3rpi ~ $ sudo su

[sudo] password for csa:

root@redhen3rpi /home/csa # cd

2. Update the repositories

root@redhen3rpi ~ # just update

3. Upgrade the packages

root@redhen3rpi ~ # just distupgrade

4. Respond to any questions during the upgrade process.

If there are configuration errors, these commands can be useful:

apt-get install --reinstall <package names>

dpkg --configure -a

apt-get -f install

apt list --upgradable

apt-get upgrade --dry-run

5. Verify

just fix-configure

just fix-install

just autoremove

6. Reboot if you're asked to, or you're installing a new kernel, or if the file system is bad

Normally, you don't need to reboot -- some Red Hen servers haven't been rebooted in more than a year.

Installing the new ffmpeg

The previous ffmpeg stopped working (segfaulted). We needed to explicitly install a series of dependencies to upgrade to ffmpeg 10:3.3.4-dmo1+deb9u1.

First put the version number in a variable:

d=10:3.3.4-dmo1+deb9u1

Second, create a temporary file listing all the dependencies in this order:

nano /tmp/ffm and paste:

libavutil55

libswresample2

libswscale4

libavcodec57

libavformat57

libavresample3

libpostproc54

libavfilter6

libavdevice57

ffmpeg

Third, run a for loop to explicitly install each of these with the version required:

for F in `cat /tmp/ffm` ; do apt-get -y install $F=$d ; done

Other scripts and packages

I also rebuilt dvb-dvbt-ts from source in the new build environment on stretch, following the instructions under the Portugal capture station.

just install dvb-dvbt-ts_0.08-1_armhf-stretch.deb

fail2ban fails to start, so copy in the new default config file:

cp /etc/fail2ban/jail.conf /etc/fail2ban/jail.local (or fail2ban fails)

I also made some script improvements on dola that need to be propagated -- here's rusalka picking them up:

./yinfo-backin dola check-cc-single-06.sh

./yinfo-backin dola cc-extract-teletext-04.sh

./yinfo-backin dola check-mpg-single-05.sh

cd /nest/cfg

HOST=dola

rsync ca:/home/$HOST/system/$HOST/nest/cfg/dvb-dvbt-ts_0.08-1_armhf-stretch.deb /nest/cfg/ -av

As user root:

cd /nest/cfg

just install dvb-dvbt-ts_0.08-1_armhf-stretch.deb

cd /nest/bin

cp -p check-cc-single-06.sh cc-extract-teletext-04.sh check-mpg-single-05.sh /usr/local/bin

cd /usr/local/bin

ln -sf check-cc-single-06.sh check-cc-single.sh

ln -sf cc-extract-teletext-04.sh cc-extract-teletext.sh

ln -sf check-mpg-single-05.sh check-mpg-single.sh

Sending the security key after install

After installing, must not forget about this:

name@cartago:~$ ssh-copy-id csa@name

and a useful quote here: "Also copied /etc/ssh/sshd_config from dola (only removed jacek from allowed users) then issued just restart ssh"