MythTV

24 Feb 2007 A while ago I had this idea of buying a new computer and making a media center out of it, using MythTV.

After many frustrating months, I managed to install ubuntu on my hardware because the kernel didn't have the right drivers...
Then after another month I manage to get the machine not to crash down on me every 2 or 3 days.
Now for the "moment supreme": the configuration of MythTV to do something useful.

Sit back and relax, this will be a funny story.

I installed MythTV even before I managed to get the machine to not crash every couple days, and I ran into the problem of XMLTV.

(BTW, I used this guide which is actually pretty good considering what a mess MythTV and XMLTV is)

Oh, where to start ?

XMLTV is a tool that fetches program schedules from the internet, parses them and dumps them in a XML format. But before it can do all that, you need to configure it of course.
XMLTV uses "grabbers" to download data from internet and parse it. There are many different grabbers, apparently one for every country code. I have to use tv_grab_be which is the grabber for Belgian programs schedules. It uses my alltime favorite http://www.teveblad.be for the schedules.

For no reason I can imagine whatsoever, before you can run this grabber, you need to configure WHICH channels you want data from (So hey, why not have an option that just downloads them all and save everyone a headache ?).

This is what such configuration looks like:

guest@berta:~$ tv_grab_be --configure
using config filename /home/guest/.xmltv/tv_grab_be.conf
A nonempty configuration file /home/guest/.xmltv/tv_grab_be.conf
already exists.  There is currently no support for altering an
existing configuration: you have to reconfigure from scratch.

Do you wish to overwrite the old configuration? [yes,no (default=no)] 

You see, I already have a configuration file and don't want to lose it (because it's hell setting it up in the first place). There is no "append to configuration" or "reconfigure". Nono, it's overwrite or don't. And consequently, if you choose not to overwrite, the program exits like this:

Exiting since you don't want to overwrite the old configuration.

So, always backup your config!

Let's see what it does when overwriting the configfile:

Do you wish to overwrite the old configuration? [yes,no (default=no)] yes
Enter the language required: [F]rench or [D]utch: D
The Default configuration for this grabber is to only grab the summary
information for programmes. (channel/start/title/brief description)

If you want detailed information (episode name, detailed description,
actors) then you must re-configure and run this grabber with the
--slow option

finding channels from teveblad.be :     teveblad.be Channel id LIFE!TV not found in channel_ids_nl file, assuming XMLTV id life!tv.teveblad.be
teveblad.be Channel id ACTUATV not found in channel_ids_nl file, assuming XMLTV id actuatv.teveblad.be
teveblad.be Channel id RTL7 not found in channel_ids_nl file, assuming XMLTV id rtl7.teveblad.be
teveblad.be Channel id S.TELEVISIE not found in channel_ids_nl file, assuming XMLTV id s.televisie.teveblad.be
teveblad.be Channel id TV5MONDE not found in channel_ids_nl file, assuming XMLTV id tv5monde.teveblad.be
teveblad.be Channel id 11 not found in channel_ids_nl file, assuming XMLTV id 11.teveblad.be
teveblad.be Channel id AB3 not found in channel_ids_nl file, assuming XMLTV id ab3.teveblad.be
teveblad.be Channel id ADULTCHANNEL not found in channel_ids_nl file, assuming XMLTV id adultchannel.teveblad.be
teveblad.be Channel id ADVENTUREONE not found in channel_ids_nl file, assuming XMLTV id adventureone.teveblad.be
teveblad.be Channel id ANIMAL not found in channel_ids_nl file, assuming XMLTV id animal.teveblad.be
teveblad.be Channel id BBCPRIME not found in channel_ids_nl file, assuming XMLTV id bbcprime.teveblad.be
teveblad.be Channel id BOOMERANG not found in channel_ids_nl file, assuming XMLTV id boomerang.teveblad.be
teveblad.be Channel id E!ENTERTAINMENT not found in channel_ids_nl file, assuming XMLTV id e!entertainment.teveblad.be
teveblad.be Channel id EURONEWS not found in channel_ids_nl file, assuming XMLTV id euronews.teveblad.be
teveblad.be Channel id EUROSPORT2 not found in channel_ids_nl file, assuming XMLTV id eurosport2.teveblad.be
teveblad.be Channel id EXTREME not found in channel_ids_nl file, assuming XMLTV id extreme.teveblad.be
teveblad.be Channel id FRANCE4 not found in channel_ids_nl file, assuming XMLTV id france4.teveblad.be
teveblad.be Channel id FRANCE5 not found in channel_ids_nl file, assuming XMLTV id france5.teveblad.be
teveblad.be Channel id HALLMARK not found in channel_ids_nl file, assuming XMLTV id hallmark.teveblad.be
teveblad.be Channel id JETIX not found in channel_ids_nl file, assuming XMLTV id jetix.teveblad.be
teveblad.be Channel id MOTORSTV not found in channel_ids_nl file, assuming XMLTV id motorstv.teveblad.be
teveblad.be Channel id NASN not found in channel_ids_nl file, assuming XMLTV id nasn.teveblad.be
teveblad.be Channel id PLAYBOYTV not found in channel_ids_nl file, assuming XMLTV id playboytv.teveblad.be
teveblad.be Channel id PRIMEACTION not found in channel_ids_nl file, assuming XMLTV id primeaction.teveblad.be
teveblad.be Channel id PRIMEFEZZTIVAL not found in channel_ids_nl file, assuming XMLTV id primefezztival.teveblad.be
teveblad.be Channel id PRIMEMOVIES not found in channel_ids_nl file, assuming XMLTV id primemovies.teveblad.be
teveblad.be Channel id PRIMEONE not found in channel_ids_nl file, assuming XMLTV id primeone.teveblad.be
teveblad.be Channel id PRIMESPORT1 not found in channel_ids_nl file, assuming XMLTV id primesport1.teveblad.be
teveblad.be Channel id REALITYTV not found in channel_ids_nl file, assuming XMLTV id realitytv.teveblad.be
got 88, done.
Enter the name of a channel, '.' to finish, '*' to list all: 

The above garbage is not the list of channels by the way, those are error messages. The real list is shown when pressing "*"

Enter the name of a channel, '.' to finish, '*' to list all: *
Known channels: 

           11
           Nickelodeon
           RTV
           WTV
           AB3
           Actua-TV
           Adult Channel
           Adventure One
           Animal Planet
           ARD
           Arte
           ATV
           AVS
           BBC1
           BBC2
           BBC Prime
           Boomerang
           BVN-TV
           Canvas
           Cartoon Network
           ClubRTL
           Discovery
           E!Entertainment
           ESPN Classic Sport
           Euronews
           CNN
           Eurosport
           Eurosport 2
           Extreme Sports Channel
           Focus
           France 2
           France 3
           France 4
           France 5
           Hallmark
           Jetix
           JIM
           KANAALTWEE
           Kanaal 3
           Kanaal Z
           Ketnet
           La Une
           La Deux
           LibertyTV
           Life!TV
           ITV - Meridian
           Motors TV
           MTV NL
           NASN
           Ned1
           Ned2
           Ned3
           Net5
           NBC/Nat.Geo.
           Playboy TV
           Prime Action
           Prime Fezztival
           Prime Movies & Series
           Prime One
           Prime Sport 1
           RAIUNO
           Reality TV
           Ring tv
           Rob tv
           RTL
           RTL4
           RTL5
           RTL7
           RTL-TVI
           S.televisie
           SBS6
           SWR
           TCM
           TF1
           TMF
           TRT
           TV Brussel
           �
           TV5Monde
           TVE
           TV-Limburg
           vijfTV
           Vitaya
           VT4
           VTM
           WDR
           BBC World
           ZDF



Enter the name of a channel, '.' to finish, '*' to list all: 

That's great! It found all TV channels!
So, let's enable 2 of my favorite channels: VT4 and Kanaal2 (because they both show decent movies and series)

Enter the name of a channel, '.' to finish, '*' to list all: ka2   
Add channel KANAALTWEE? [yes,no (default=yes)] yes
Enter the name of a channel, '.' to finish, '*' to list all: vt4
Add channel VT4? [yes,no (default=yes)] yes
Enter the name of a channel, '.' to finish, '*' to list all: 

"Hey! You used "ka2" to select KANAALTWEE!"
Yes, that's right ! I can use abbreviations! But I didn't know about this untill I accidently typed it in. You see, xmltv has a file in /usr/share/xmltv/ for every grabber, and that file contains mappings from channel names to URL's and more info about channels.

"OK, but wait: why do you have to confirm every time ? Can you use wildcards or something to select a group of channels at once ?"
Hahahaha so naive! No you can't!
See ? This is what happens if I want to be more ambiguous about the channel name, in the hope it will select multiple channels:

Enter the name of a channel, '.' to finish, '*' to list all: v
'v' matches lots of channels, be more specific.
Enter the name of a channel, '.' to finish, '*' to list all: 

For what it's worth: there are only 4 channels that start with "v"...

Selecting multiple channels... This actually looks like a good use for checkboxes in a GUI, so what about it ?

There is this mention of a GUI in the manpage of tv_grab_be:

       --gui OPTION Use this option to enable a graphical interface to be used.  OPTION may be
       ’Tk’, or left blank for the best available choice.  Additional allowed values of OPTION
       are ’Term’ for normal terminal output (default) and ’TermNoProgressBar’ to disable the
       use of Term::ProgressBar.

But this option seems not to do anything at all: surprise !
I didn't give up here though. I actually found some program somewhere that does all this stuff with a GUI, but instead of using the GUI like it should, it just opens a window with the list of channel and then ASKS in an inputfield what channel to enable.

Bah! Down with this junk.
It's actually easier to edit the generated configfile manually

This will select all channels from "channels_ids_nl", the ones tv_grab_be knows about, and converts them into configuration file lines.

cat /usr/share/xmltv/tv_grab_be/channel_ids_nl |grep -v "^#"|sed 's/:.*/ nl dodetail/' | sed 's:^:channel :'

If you copy paste all the lines tv_grab_be outputs about channels that it doesn't know about (like the error messages I pasted above) into a file called "output", then the following script will make them into configurationfile lines aswell:

cat output |grep "not found in channel_ids_nl"|sed 's:.* :channel :'|sed 's:$: nl dodetail:'

This is what my configuration file looks like:

#
# tv_grab_be config file.
#
# Format is:
# language 
# detailstarttime <24hr clock time>
# detailstoptime <24hr clock time>
# detailgenre     #- may be repeated mutiple times
# channel   [dodetail] #- may be repeated mutiple times
# 

#
# definition of language mode: fr or nl
#
language nl

#
# definition of start and stop times for retrieving detailed
# information for programmes. Times must be in 24 hour clock
# and may overlap a day bounday (eg 17:00 - 02:00)
#
detailstarttime 05:00
detailstoptime 02:00

# 
# definition of genres/category substrings to get detailed information for
# multiple detailgenre lines can be defined 
# eg: *ALL* -- match all categories (use with care!)
#    *NONE* -- match completely blank categories.
#    FILM -- will match tvfilm, telefilm, as well as film
#
detailgenre *ALL*

# 
# definition of channels to grab, and whether to grab detailed info for the channel
# multiple channel lines can be defined as:
#   channel xmltv.channel.id language dodetail
# or (if no detail required)
#   channel xmltv.channel.id
#
# where language is fr or nl (for future use: currently ignored) 
#
channel Nickelodeon.nl nl dodetail
channel RTV.be nl dodetail
channel ard.daserste.de nl dodetail
channel arte.arte-tv.com nl dodetail
channel atv.be nl dodetail
channel avs.be nl dodetail
channel bbc1.bbc.co.uk nl dodetail
channel bbc2.bbc.co.uk nl dodetail
channel blauw.cplus.be nl dodetail
channel bvn.nl nl dodetail
channel canalplusfr.cplus.be nl dodetail
channel canvas.vrt.be nl dodetail
channel cartoonnetwork.nl nl dodetail
channel clubrtl.rtl.be nl dodetail
channel europe.cnbc.com nl dodetail
channel europe.cnn.com nl dodetail
channel eurosport.com nl dodetail
channel france2.fr nl dodetail
channel france3.fr nl dodetail
channel jim.be nl dodetail
channel ka2.be nl dodetail
channel kanaal3.be nl dodetail
channel kanaalz.be nl dodetail
channel ketnet.vrt.be nl dodetail
channel la1.rtbf.be nl dodetail
channel la2.rtbf.be nl dodetail
channel libertytv.com nl dodetail
channel meridiantv.co.uk nl dodetail
channel ned1.omroep.nl nl dodetail
channel ned2.omroep.nl nl dodetail
channel ned3.omroep.nl nl dodetail
channel net5.sbsnet.nl nl dodetail
channel ngc.nationalgeographic.com nl dodetail
channel raiuno.rai.it nl dodetail
channel ringtv.be nl dodetail
channel robtv.robnet.be nl dodetail
channel rood.cplus.be nl dodetail
channel rtl.de nl dodetail
channel rtl4.rtl.nl nl dodetail
channel rtl5.rtl.nl nl dodetail
channel rtltvi.rtl.be nl dodetail
channel sbs6.sbsnet.nl nl dodetail
channel swr.de nl dodetail
channel tf1.fr nl dodetail
channel tmf.be nl dodetail
channel trt.net.tr nl dodetail
channel tv-brussel.be nl dodetail
channel tv1.vrt.be nl dodetail
channel tv5.org nl dodetail
channel tveinter.rtve.es nl dodetail
channel tvl.be nl dodetail
channel vitaya.tv nl dodetail
channel vt4.be nl dodetail
channel vtm.vrt.be nl dodetail
channel wdr.de nl dodetail
channel world-tv.bbc.com nl dodetail
channel yorin.nl nl dodetail
channel zdf.de nl dodetail
channel zones.tv nl dodetail
channel tcm.teveblad.be nl dodetail
channel focus.teveblad.be nl dodetail
channel WTV.teveblad.be nl dodetail
channel mtv.nl nl dodetail
channel vijftv.be nl dodetail
channel discoveryeurope.com nl dodetail
channel espnclassicsport.com nl dodetail
channel life!tv.teveblad.be nl dodetail
channel actuatv.teveblad.be nl dodetail
channel rtl7.teveblad.be nl dodetail
channel s.televisie.teveblad.be nl dodetail
channel tv5monde.teveblad.be nl dodetail
channel 11.teveblad.be nl dodetail
channel ab3.teveblad.be nl dodetail
channel adultchannel.teveblad.be nl dodetail
channel adventureone.teveblad.be nl dodetail
channel animal.teveblad.be nl dodetail
channel bbcprime.teveblad.be nl dodetail
channel boomerang.teveblad.be nl dodetail
channel e!entertainment.teveblad.be nl dodetail
channel euronews.teveblad.be nl dodetail
channel eurosport2.teveblad.be nl dodetail
channel extreme.teveblad.be nl dodetail
channel france4.teveblad.be nl dodetail
channel france5.teveblad.be nl dodetail
channel hallmark.teveblad.be nl dodetail
channel jetix.teveblad.be nl dodetail
channel motorstv.teveblad.be nl dodetail
channel nasn.teveblad.be nl dodetail
channel playboytv.teveblad.be nl dodetail
channel primeaction.teveblad.be nl dodetail
channel primefezztival.teveblad.be nl dodetail
channel primemovies.teveblad.be nl dodetail
channel primeone.teveblad.be nl dodetail
channel primesport1.teveblad.be nl dodetail
channel realitytv.teveblad.be nl dodetail

Just comment out the channels you don't want data from, by placing a # in front of the line

Now that the configuration file is adequate, you can fetch the data and store it in a file, as a test:

tv_grab_be --slow --output /tmp/whatever

This can take a while...
Once the file is ready, you can insert it into the mythtv database with

mythfilldatabase --file 1 -1 /tmp/whatever

Mythfilldatabase has no manpage by the way, but you can check it's option with "mythfilldatabase --help"

Right! That's the XMLTV part.
What about MythTV ?
Well, following a complicated setup procedure using weird terminology and many many useless(?) default settings (Hey, what's up with all these settings ? Do I even need those ? It's confusing as hell) you should be able to register a TV-card, and hook it up to an XMLTV program schedule.

In my case, I had to select Belgian TV schedules (Dutch) which apparently uses the tv_grab_be_tvb command. Of course, this command doesn't exist... I have tv_grab_be, but that is not listed in the mythtv config screen. Great.

After some digging around, I found out that this setting is stored in the "videosource" MySQL table. I set it to "tv_grab_be" instead, but it still wouldn't work. Next, I added an alias for tv_grab_be_tvb that points to tv_grab_be, but mythtv ignores it...
So finally, I symlinked tv_grab_be_tvb to tv_grab_be in /usr/bin/

I was instructed to run "mythfilldatabase --manual", but that coredumped. That's some really stable piece of software we have here...

Oh by the way, I just realized that the steps I'm complaining about are not really in the order I did them in the first place. Before mythfilldatabase actually succeeded, I believe I had to make another symlink because mythtv didn't find the xmltv configuration file.

If you have the same problem, look at the output of mythfilldatabase. It should say something like "file /home/blabla...whatever not found. Run me with --configure"

That error comes from tv_grab_be, which can't locate the file that mythtv passes to it. In my case the file was /home/guest/.mythtv/pvr500-1.xmltv and I symlinked that to /home/guest/.xmltv/tv_grab_be.conf

guest@berta:~/.mythtv$ ln -s ../.xmltv/tv_grab_be.conf pvr500-1.xmltv

MythTV has the ability to scan for channels and store the result in it's database.
But the names of the channels are all wrong. I looked up the channel frequencies once and made a bunch of scripts to tune my TV-card from the commandline, by using the channel name as a command.

I got the frequencies here

After the scan was complete, there were 67 channels in the database.
So now comes the task to rename all those channel names...
It occurs to me that there probably needs to be a link between the MythTV channel names and those used in XMLTV. Although I'm nore sure what the link is yet.

...

It appears that XMLTV fills in the channel database itself (or that's what mythtv claims).
I deleted all channels and tried adding the xmltv output file again. It wouldn't load because the backend wasn't running. And that backend wasn't running because the storage directory (where it records all shows in) wasn't writable by whatever user the backend runs as.

After fixing that, I was able to watch TV, but the channels were only showing static and only 1 channel was in the program schedule of MythTV.

In the mythtv-setup program, there is a menu "Input Connections" where there is a setting "fetch channels from listing", which appears to do something ?

Anyway, I had to run "/etc/cron.daily/mythtv_backend", which complained that it couldn't find the xmltv configfile.
A closer look in that file revealed that is was trying to run as user mythtv, which I don't use. So I had to change it to "guest"

...

That still didn't fix anything.

Dumping the database shows that for some reason, Kanaal2 now has no freqid (the frequency) and that the channel number (E23) is set to Ka2. Of course this won't work...
I manually reset it to the right settings and now this channel works!

You can also change the channel settings by pressing "e" while watching it. I wonder what other keys there are...