CRIDs
Re: CRIDs
I think the CRID data, in itself, is essentially useless. As mentioned earlier, I'll extract the information I can, adding it to my Single EPG info and the EPG search results. I'll push the code, but don't intend to release it.
Re: CRIDs
Hi Adoxa,
Regards,
Ian.
I agree. The initial value in the CRID data is as a point of comparison between events for the purpose of trying to more reliably identify separate programs by a method other than the broadcast title (which often contains obfuscating data). Also to try and identify new or repeat episodes of an individual program.
Understood. I think I am beginning to understand your development style.
Regards,
Ian.
-
- Wizard God
- Posts: 32714
- Joined: Tue Sep 04, 2007 13:49
- Location: Canberra; Black Mountain Tower transmitters
Re: CRIDs
What would that be? Should it, for example, include returning the Content Identifier Table so that CRIDs with their location specified as "Carried in Content Identifier Table" can be constructed? Tables of the network-level and transport stream-level and sevice-level default authorities for abbreviated CRIDs?
The current implementation of the Python interface simply returns a list of tuples representing eCridData structures and containing fully-specified CRIDs for an event as a list of text strings. I'm not sure whether there's any real need for the CRID components to be displayed. I'm not even convinced there's a need for a Converter for CRIDs at all. If the Python code wants to split the CRID into its logical component parts, there's always:
Code: Select all
>>> import urlparse
>>> urlparse.urlsplit("crid://canberra.nitv.sbs.au/211351#IM")
SplitResult(scheme='crid', netloc='canberra.nitv.sbs.au', path='/211351', query='', fragment='IM')
>>>
Peter
T4 HDMI
U4, T4, T3, T2, V2 test/development machines
Sony BDV-9200W HT system
LG OLED55C9PTA 55" OLED TV
T4 HDMI
U4, T4, T3, T2, V2 test/development machines
Sony BDV-9200W HT system
LG OLED55C9PTA 55" OLED TV
-
- Wizard God
- Posts: 32714
- Joined: Tue Sep 04, 2007 13:49
- Location: Canberra; Black Mountain Tower transmitters
Re: CRIDs
Why not use the existing API and reformat the CRID into the form you want:
Code: Select all
def getCrid(self, event, types):
crids = event.getCridData(types)
if not crids or not crids[0][2]:
return ""
try:
import urlparse
fields = urlparse.urlsplit(crids[0][2])
ret = fields.netloc + fields.path
print "[EventName] getCrid", ret
return ret
except:
return "unparseable CRID"
Code: Select all
{1178}< 442.444> [EventName] getCrid canberra2.abc.net.au/NC1823C
{1178}< 442.445> [EventName] getCrid canberra2.abc.net.au/NC1823C040S00
Peter
T4 HDMI
U4, T4, T3, T2, V2 test/development machines
Sony BDV-9200W HT system
LG OLED55C9PTA 55" OLED TV
T4 HDMI
U4, T4, T3, T2, V2 test/development machines
Sony BDV-9200W HT system
LG OLED55C9PTA 55" OLED TV
Re: CRIDs
Here's my CRID code (and its Full-Metal-Wizard skin).
Here's an EPG search showing the CRID for each channel, as well as in the info (if there's no CRID the icons are flush on the right):
Event info adds it to the name:
The movie list places it to the right of the genre:
IanSav, if you'd like to point your Seven contact here, maybe they can tell me what I missed.
Here's an EPG search showing the CRID for each channel, as well as in the info (if there's no CRID the icons are flush on the right):
Event info adds it to the name:
The movie list places it to the right of the genre:
IanSav, if you'd like to point your Seven contact here, maybe they can tell me what I missed.
Re: CRIDs
Hi Adoxa,
I will reach out to him tomorrow. I doubt he will look at your code (he is an engineer and not a programmer). I am also not sure how much information he will be allowed to give me. I'll see what I can find out.
If I have enough time I will try to reach out to my SBS contact to see why they have all the different authorities. It would be good if SBS joined Seven and Nine with a single authority across all their LCNs. I have a friend who works at the ABC. He was in TV but is now in radio. He may have an old contact that can help me with the same question for the ABC.
I don't think I have any contacts within Ten.
Regards,
Ian.
I will reach out to him tomorrow. I doubt he will look at your code (he is an engineer and not a programmer). I am also not sure how much information he will be allowed to give me. I'll see what I can find out.
If I have enough time I will try to reach out to my SBS contact to see why they have all the different authorities. It would be good if SBS joined Seven and Nine with a single authority across all their LCNs. I have a friend who works at the ABC. He was in TV but is now in radio. He may have an old contact that can help me with the same question for the ABC.
I don't think I have any contacts within Ten.
Regards,
Ian.
Re: CRIDs
Hi,
I contacted my ABC friend. He is not involved with areas that could be of assistance to this matter. He is going to see if he can find some appropriate contacts for me. No promises.
By the way, has anyone looked at the CRIDs for DVB-T radio? ABC has no CRID data at all. SBS has very strange looking Episode CRID data only.
Should the DVB-T radio stations have CRID data?
Regards,
Ian.
I contacted my ABC friend. He is not involved with areas that could be of assistance to this matter. He is going to see if he can find some appropriate contacts for me. No promises.
By the way, has anyone looked at the CRIDs for DVB-T radio? ABC has no CRID data at all. SBS has very strange looking Episode CRID data only.
Should the DVB-T radio stations have CRID data?
Regards,
Ian.
-
- Wizard God
- Posts: 32714
- Joined: Tue Sep 04, 2007 13:49
- Location: Canberra; Black Mountain Tower transmitters
Re: CRIDs
SBS has completely non-conforming CRIDs where what looks like it might be Episode CRID path components have been put into the authority part of the CRID, and the path part of the CRID has been left empty.
From Grumpy_Geoff's data: SBS Radio 1 14.08, 21:00 crid://8931f97735014d348c16eb619aee552a Korean
Code: Select all
>>> from urlparse import urlsplit
>>> urlsplit("crid://8931f97735014d348c16eb619aee552a")
SplitResult(scheme='crid', netloc='8931f97735014d348c16eb619aee552a', path='', query='', fragment='')
Also, according to Freeview OP-72, the maximum permitted length of the data part of the CRID is 29 characters, including the leading "/". If the SBS Radio CRIDs were to be re-written to have a sensible authority and use the existing "authority" as the data part, the data part is longer than permitted in OP-72 - it's 33 characters including the "/".
Good question, but no matter what the answer to that question is, they shouldn't be including non-conforming CRIDs in any service.
On the original question, as far as I can see, OP-72 doesn't mention radio services either way.
More naughty words that are censored by Sucuri: data:
Peter
T4 HDMI
U4, T4, T3, T2, V2 test/development machines
Sony BDV-9200W HT system
LG OLED55C9PTA 55" OLED TV
T4 HDMI
U4, T4, T3, T2, V2 test/development machines
Sony BDV-9200W HT system
LG OLED55C9PTA 55" OLED TV
Re: CRIDs
Hi,
I had a long chat with my contact at Seven. The CRID encoding is not done directly by Seven but is an output of a black box system that Seven uses to generate their operational data. Thus, the encoding is the intellectual property of an organisation outside of the Network. I was reminded that we don't need to know what the CRIDs are but simply use them as tokens for comparison to see if one event relates in any way to another even. There is one issue that needs to be noted. When these systems are changed by the broadcasters the CRID encodings *can* change! This happened to Seven when they upgraded equipment from one manufacturer to that of another.
My contact also indicated that Seven intends their CRIDs to be applicable across all their LCNs. While the aim was to have the CRIDs unique across ALL broadcasters this has not happened. I was cautioned that while the intention for Seven is for the CRIDs to be uniform across all the Seven LCNs he had not specifically verified that this is the case.
I raised the issue of the differing authorities by some broadcasters. The suggestion was that this should be a difference that can be ignored. This is not an issue for Seven as they use a single authority. It was suggested that when we code the series recording capability into Enigma2 we add a configuration option to allow for minor differences in the authority to be ignored. I will take this up with SBS when I contact them.
My contact has no involvement with Radio LCNs so had no opinion as to whether or not they should have CRIDs. He asked if anyone would record a radio program. I said it was possible and that the Radio LCNs do have an EPG and there are events that can be separately identifiable that can also repeat. This is what CRIDs are used to identify. I will also ask this question of SBS when I contact them.
Is there anything else you want me to find out?
Regards,
Ian.
I had a long chat with my contact at Seven. The CRID encoding is not done directly by Seven but is an output of a black box system that Seven uses to generate their operational data. Thus, the encoding is the intellectual property of an organisation outside of the Network. I was reminded that we don't need to know what the CRIDs are but simply use them as tokens for comparison to see if one event relates in any way to another even. There is one issue that needs to be noted. When these systems are changed by the broadcasters the CRID encodings *can* change! This happened to Seven when they upgraded equipment from one manufacturer to that of another.
My contact also indicated that Seven intends their CRIDs to be applicable across all their LCNs. While the aim was to have the CRIDs unique across ALL broadcasters this has not happened. I was cautioned that while the intention for Seven is for the CRIDs to be uniform across all the Seven LCNs he had not specifically verified that this is the case.
I raised the issue of the differing authorities by some broadcasters. The suggestion was that this should be a difference that can be ignored. This is not an issue for Seven as they use a single authority. It was suggested that when we code the series recording capability into Enigma2 we add a configuration option to allow for minor differences in the authority to be ignored. I will take this up with SBS when I contact them.
My contact has no involvement with Radio LCNs so had no opinion as to whether or not they should have CRIDs. He asked if anyone would record a radio program. I said it was possible and that the Radio LCNs do have an EPG and there are events that can be separately identifiable that can also repeat. This is what CRIDs are used to identify. I will also ask this question of SBS when I contact them.
Is there anything else you want me to find out?
Regards,
Ian.
-
- Wizard God
- Posts: 32714
- Joined: Tue Sep 04, 2007 13:49
- Location: Canberra; Black Mountain Tower transmitters
Re: CRIDs
Does your contact have any information about Prime CRIDs? Prime Canberra used to carry CRIDs (e.g. in Nov 2016), but no longer appears to. When Prime Canberra carried CRIDs, they appear to have been in an obfuscated form similar to that used by Seven's CRIDs now. E.g.:
Also. when Prime Canberra carried CRIDs, they weren't under a single authority name:
can.6.primetv.com.au PRIME7 Canberra
can.62.primetv.com.au 7TWO Canberra
can.63.primetv.com.au 7mate Canberra
can.65.primetv.com.au ishoptv
can.68.primetv.com.au RACING.COM
Code: Select all
crid 0946:099e:3281 episode 2016-11-22T12:01 <crid://can.6.primetv.com.au/qufquwemda4> <PRIME7 Canberra> <Jack Reed: Death And Vengeance>
can.6.primetv.com.au PRIME7 Canberra
can.62.primetv.com.au 7TWO Canberra
can.63.primetv.com.au 7mate Canberra
can.65.primetv.com.au ishoptv
can.68.primetv.com.au RACING.COM
Peter
T4 HDMI
U4, T4, T3, T2, V2 test/development machines
Sony BDV-9200W HT system
LG OLED55C9PTA 55" OLED TV
T4 HDMI
U4, T4, T3, T2, V2 test/development machines
Sony BDV-9200W HT system
LG OLED55C9PTA 55" OLED TV
Re: CRIDs
That would explain why RACING.COM is different.
It does look like the Seven series CRID is consistent across their LCNs. But as a series identifier, it's not directly comparable as it usually includes a season identifier. Some ids are only four characters, but that's not enough to uniquely identify a series (e.g. Life Off Road and The Lion Guard both have the same first four characters). Harry's Practice is one instance where even the first six characters are different.My contact also indicated that Seven intends their CRIDs to be applicable across all their LCNs. [...] I was cautioned that while the intention for Seven is for the CRIDs to be uniform across all the Seven LCNs he had not specifically verified that this is the case.
For my purposes I just test the suffix (e.g. .sbs.au), but that might not work nation-wide.I raised the issue of the differing authorities by some broadcasters. The suggestion was that this should be a difference that can be ignored.
I'm good, thanks.Is there anything else you want me to find out?
Re: CRIDs
Hi Prl,
I also received an SMS from my friend in the ABC this afternoon. He may have a contact for someone who can help with CRID information. If he is permitted to talk to people outside of the ABC I have arranged for them to get my telephone number. This may get us some information about the variable authorities and the applicability of CRIDs to radio broadcasts.
My contact at SBS wasn't answering their phone this afternoon. I will try again next week.
Regards,
Ian.
I will EMail this as a follow up question.prl wrote: ↑Fri Aug 17, 2018 16:14Does your contact have any information about Prime CRIDs? Prime Canberra used to carry CRIDs (e.g. in Nov 2016), but no longer appears to. When Prime Canberra carried CRIDs, they appear to have been in an obfuscated form similar to that used by Seven's CRIDs now. E.g.:Also. when Prime Canberra carried CRIDs, they weren't under a single authority name:Code: Select all
crid 0946:099e:3281 episode 2016-11-22T12:01 <crid://can.6.primetv.com.au/qufquwemda4> <PRIME7 Canberra> <Jack Reed: Death And Vengeance>
can.6.primetv.com.au PRIME7 Canberra
can.62.primetv.com.au 7TWO Canberra
can.63.primetv.com.au 7mate Canberra
can.65.primetv.com.au ishoptv
can.68.primetv.com.au RACING.COM
I also received an SMS from my friend in the ABC this afternoon. He may have a contact for someone who can help with CRID information. If he is permitted to talk to people outside of the ABC I have arranged for them to get my telephone number. This may get us some information about the variable authorities and the applicability of CRIDs to radio broadcasts.
My contact at SBS wasn't answering their phone this afternoon. I will try again next week.
Regards,
Ian.
Re: CRIDs
Hi,
The next question will become who wants to tackle changing AutoTimers to add the option of using CRID data to create the series link functionality with repeat event filtering. I will call SHOTGUN and say NOT ME! To be sure I will also take a preemptive step back.
By the way, if anyone looks at AutoTimers can they please consider looking at all timers with a view to capturing the start and end padding so that event details can be better calculated.
Regards,
Ian.
The next question will become who wants to tackle changing AutoTimers to add the option of using CRID data to create the series link functionality with repeat event filtering. I will call SHOTGUN and say NOT ME! To be sure I will also take a preemptive step back.
By the way, if anyone looks at AutoTimers can they please consider looking at all timers with a view to capturing the start and end padding so that event details can be better calculated.
Regards,
Ian.
-
- Wizard God
- Posts: 32714
- Joined: Tue Sep 04, 2007 13:49
- Location: Canberra; Black Mountain Tower transmitters
Re: CRIDs
That's probably an even bigger can of worms than using CRIDs in AutoTimer.
I've had no problems getting some Beyonwiz-specific code past the gatekeepers of the enigma2-plugins repository (specifically adding IceTV timer icons to the EPGSearch search results). Of course, with suitable protections so that the code worked on other enigma2 systems.
Peter
T4 HDMI
U4, T4, T3, T2, V2 test/development machines
Sony BDV-9200W HT system
LG OLED55C9PTA 55" OLED TV
T4 HDMI
U4, T4, T3, T2, V2 test/development machines
Sony BDV-9200W HT system
LG OLED55C9PTA 55" OLED TV
-
- Wizard God
- Posts: 32714
- Joined: Tue Sep 04, 2007 13:49
- Location: Canberra; Black Mountain Tower transmitters
Re: CRIDs
IanSav wrote: ↑Fri Aug 17, 2018 17:02The next question will become who wants to tackle changing AutoTimers to add the option of using CRID data to create the series link functionality with repeat event filtering. I will call SHOTGUN and say NOT ME! To be sure I will also take a preemptive step back.
I'm not convinced that it would be all that much better than using AutoTimers as they are now. Things like the fact that the ABC encodes the season number into their series CRIDs will make CRID-based AutoTimers fail in ways that are at least as confusing as the ways that current AutoTimers fail.
In the ACT, only 3 of the 5 broadcasters have CRID coverage, anyway (ABC, SBS & SCA).
OP-72 just doesn't specify well enough what the CRIDs should represent. And, of course, some broadcasters just ignore bits of it anyway, or can't be bothered at all. The fact that Prime Canberra once carried CRIDs and no longer does, and the various other problems inconsistencies and lack of data don't give me the impression of an industry determined to make CRIDs work.
Peter
T4 HDMI
U4, T4, T3, T2, V2 test/development machines
Sony BDV-9200W HT system
LG OLED55C9PTA 55" OLED TV
T4 HDMI
U4, T4, T3, T2, V2 test/development machines
Sony BDV-9200W HT system
LG OLED55C9PTA 55" OLED TV
-
- Wizard God
- Posts: 32714
- Joined: Tue Sep 04, 2007 13:49
- Location: Canberra; Black Mountain Tower transmitters
Re: CRIDs
This continues new discussion of the use of CRIDs to improve the reliability of AutoTimers that was started here.
I was fairly sure that I'd remembered correctly, but thanks for the link to your summary.
Properly implemented CRIDs would probably be a good resource for AutoTimers. What we have, much less so.
If there's anyone who's still enthusiastic about using CRIDs to make AutoTimers better, I'm happy to get my CRID extraction code working again and give it to them. NB: the main works of it are C++, with a few Python hooks so that CRIDs can be displayed in the GUI. I don't think that there's any call for displaying them, but IIRC, IanSav thinks there is.
But here's a run-down of the reasons why I don't think that CRIDs are useful for AutoTimers. Last time I looked at CRIDs:
I was fairly sure that I'd remembered correctly, but thanks for the link to your summary.
Properly implemented CRIDs would probably be a good resource for AutoTimers. What we have, much less so.
If there's anyone who's still enthusiastic about using CRIDs to make AutoTimers better, I'm happy to get my CRID extraction code working again and give it to them. NB: the main works of it are C++, with a few Python hooks so that CRIDs can be displayed in the GUI. I don't think that there's any call for displaying them, but IIRC, IanSav thinks there is.
But here's a run-down of the reasons why I don't think that CRIDs are useful for AutoTimers. Last time I looked at CRIDs:
- As adoxa details in the post he linked to, show CRIDs don't stay the same for different seasons of the same series.
- There are multiple naming authorities for CRID data, and series and episode CRIDs are not consistent between different broadcasters.
- For some broadcasters, there are multiple naming authorities for the same broadcaster, though it's possible that the naming is consistent between different naming authorities for the same broadcaster, though there is no reason to expect that they would be the same. E.g, for ABC Perth: crid://perth2.abc.net.au, crid://perth20.abc.net.au, crid://perth21.abc.net.au, crid://perth22.abc.net.au, crid://perth23.abc.net.au, crid://perthabcnews24.abc.net.au.
- According to FreeTV OP 72, broadcasters are supposed to use a private data specifier in the range 0x3200-0x320f to flag that the same loop contains CRID data. Seven Perth uses 0x233a and Prime7 Canberra uses 0x0029. Those values appear to already be allocated to other broadcasters.
- In Canberra, WIN carries almost no CRID data.
- Some SBS radio services use CRIDs that are in a non-compliant form (e.g. crid://9cc19ee1-53a4-473d-ba02-21a7b5d82ed8). They are not syntactically valid CRIDs, and even if they were, the part that is apparently supposed to be the data part of the CRID (the /9cc19ee1-53a4-473d-ba02-21a7b5d82ed8) is longer (37 characters) than OP 72 allows (29 characters).
- The FreeTV OP 72 spec allows for the CRID default authority (DA) (used for compact CRIDs that don't have an authority) to be in the SDT, NIT and the Transport Stream loop of the NIT. I have coded all of those, but as far as I can see, only the SDT and NIT TS used for DAs, so the NIT code remains untested. The DVB standard also allows DAs in the BAT. The Beyonwiz enigma2 code has a hook for processing the BAT, but it's unused in Australian broadcasts, so I have no way of testing DA extraction code for the BAT.
- It would be preferable to have CRIDs used in the oe-alliance enigma2-plugins AutoTimers, but that would require the CRID capability to be in all enigma2 images that continue to cherry-pick from the oe-alliance repository. However, I have no way to test anything to do with CRIDs and their use outside Australia.
- A single naming authority for the whole scope of the EPG data.
- Consistent series CRID naming for different seasons of the same series.
- Historically consistent naming of both series and episode CRIDs, so that long-term series AutoTimers can properly follow seasons of the same series (also needs the point above), and so that CRIDs can potentially be saved to allow repeat detection in the absence of any consistent repeat metadata in the rest of the EPG. Even with that implementation might be a bit tricky (e.g. handling cancelled shows).
- All services to carry full CRID data.
- Have all the other inconsistencies and errors cleaned up.
Peter
T4 HDMI
U4, T4, T3, T2, V2 test/development machines
Sony BDV-9200W HT system
LG OLED55C9PTA 55" OLED TV
T4 HDMI
U4, T4, T3, T2, V2 test/development machines
Sony BDV-9200W HT system
LG OLED55C9PTA 55" OLED TV
-
- Wizard God
- Posts: 32714
- Joined: Tue Sep 04, 2007 13:49
- Location: Canberra; Black Mountain Tower transmitters
Re: CRIDs
I still have my most recent implementation of my CRID extraction code. If anyone is seriously interested in implementing CRID matching in AutoTimers, I'm willing to get that code running again and give it to them as a set of diffs that can be applied using patch.
Implementing that would require knowledge of C++ and Python, and the willingness to grapple with the morass that is the AutoTimer code.
Getting it all accepted into other enigma2 images would require quite a bit of research into how CRIDs are used (or not) in other markets, and possibly a not inconsiderable diplomatic effort.
Personally, I'm with peteru's comment up-thread that it simply isn't worth the effort (he worded it rather more strongly than I did )
Implementing that would require knowledge of C++ and Python, and the willingness to grapple with the morass that is the AutoTimer code.
Getting it all accepted into other enigma2 images would require quite a bit of research into how CRIDs are used (or not) in other markets, and possibly a not inconsiderable diplomatic effort.
Personally, I'm with peteru's comment up-thread that it simply isn't worth the effort (he worded it rather more strongly than I did )
Peter
T4 HDMI
U4, T4, T3, T2, V2 test/development machines
Sony BDV-9200W HT system
LG OLED55C9PTA 55" OLED TV
T4 HDMI
U4, T4, T3, T2, V2 test/development machines
Sony BDV-9200W HT system
LG OLED55C9PTA 55" OLED TV
-
- Wizard God
- Posts: 32714
- Joined: Tue Sep 04, 2007 13:49
- Location: Canberra; Black Mountain Tower transmitters
Re: CRIDs
True, for CRIDs from some broadcasters in the AU market. All bets are off for any other market.
There's no specification in the relevant DVB standard (ETSI TS 102 323) as to the internal structure of the data part of a CRID, other than the syntax to mark the IMI part. SBS chose random UUIDs for what seems to be intended to be the data part of the CRIDs its radio services, for example.
Peter
T4 HDMI
U4, T4, T3, T2, V2 test/development machines
Sony BDV-9200W HT system
LG OLED55C9PTA 55" OLED TV
T4 HDMI
U4, T4, T3, T2, V2 test/development machines
Sony BDV-9200W HT system
LG OLED55C9PTA 55" OLED TV
-
- Wizard God
- Posts: 32714
- Joined: Tue Sep 04, 2007 13:49
- Location: Canberra; Black Mountain Tower transmitters
Re: CRIDs
I have my experimental code to handle CRIDs working in firmware 20191106.
If anyone is really interested in trying to implement the use of CRIDs in AutoTimers (or for anything else), I'm happy to post the diffs here.
The diffs include code and changes the easy-skin-aus-hd skin to display CRIDs in the EPG and in recordings in the media selection screen, provided the EPG entry for the program has CRID entries. The code changes for EventName converter used to display the CRIDs in the UI show an example of how to access an event's CRID data.
If anyone is really interested in trying to implement the use of CRIDs in AutoTimers (or for anything else), I'm happy to post the diffs here.
The diffs include code and changes the easy-skin-aus-hd skin to display CRIDs in the EPG and in recordings in the media selection screen, provided the EPG entry for the program has CRID entries. The code changes for EventName converter used to display the CRIDs in the UI show an example of how to access an event's CRID data.
Peter
T4 HDMI
U4, T4, T3, T2, V2 test/development machines
Sony BDV-9200W HT system
LG OLED55C9PTA 55" OLED TV
T4 HDMI
U4, T4, T3, T2, V2 test/development machines
Sony BDV-9200W HT system
LG OLED55C9PTA 55" OLED TV