2016 HTPC for Catfisch Cinema

Discussion in 'Home Theater PCs' started by DaveF, Aug 6, 2017.

  1. jcroy

    jcroy Cinematographer

    Joined:
    Nov 28, 2011
    Messages:
    2,769
    Likes Received:
    771
    Trophy Points:
    4,110
    Real Name:
    jr
    For example, there's a few reports of Sony's bd-mod releases where the alleged "burned" bd-r discs have deliberate bad sectors protection drm on them. (Allegedly aacs might not be used on such bd-mod burned discs, though not definitive).

    Nevertheless, it is a big enough issue that the anydvd folks have taken account of it already.

    (I won't bother with any links).
     
  2. DaveF

    DaveF Moderator
    Moderator

    Joined:
    Mar 4, 2001
    Messages:
    18,612
    Likes Received:
    1,754
    Trophy Points:
    9,110
    Location:
    One Loudoun, Ashburn, VA
    Real Name:
    David Fischer
    I've seen some discs with the multitude of playlists. Very obnoxious. Fortunately, many discs remain quite straightforward.

    I'm on a buying semi-hiatus right now. I've got more movies than time to watch.
     
  3. jcroy

    jcroy Cinematographer

    Joined:
    Nov 28, 2011
    Messages:
    2,769
    Likes Received:
    771
    Trophy Points:
    4,110
    Real Name:
    jr
    Definitely very annoying.

    (From an historical perspective).

    This idea of multiple playlists + obfuscation drm has had a long tradition in dvd. For example, many Disney 2009-2010 dvd titles and Paramount 2009-2014 dvd titles already used it. Many Lionsgate dvd titles have used it since mid- 2013. Though what made it "manageable" and relatively easy to figure out for dvd, is that there appears to be at most a maximum of 99 playlists allowed in the dvd spec limitations. (I strongly suspect this is the primary reason why anydvd/dvdfab defeated the dvd's multiple playlist + obfuscation basketcase drm a long time ago).

    In contrast for the case of bluray, that maximum number of playlists is much larger than 99, which makes it a lot harder to figure out by hand which playlists are junk and which ones are not. It is basically the same principle as the dvd's older variant of multiple playlists + obfuscation, but without dvd's spec limitations and cranked up on steroids.
     
  4. 24 Aug 11, 2017
    Last edited: Aug 11, 2017
    jcroy

    jcroy Cinematographer

    Joined:
    Nov 28, 2011
    Messages:
    2,769
    Likes Received:
    771
    Trophy Points:
    4,110
    Real Name:
    jr
    The way I think about extra basketcase dvd drm nowadays, is how it foreshadows many possible bluray drm schemes which Sony and other drm designers can come up with.

    For example, such as the multiple playlists + obfuscation "screenpass" drm on bluray, which had many antecedents back in the late-2000s on dvd. With the dvd's severe specs limitations lifted, this multiple playlist obfuscation is viable nowadays on bluray. Back in the dvd era, this multiple playlist obfuscation was easy and fast enough to defeat even by a sheer brute force method of going through every single playlist and seeing which ones looked "sane".

    In contrast, going through hundreds or thousands of playlists is simply not viable to do by hand. But going through less than 99 playlists is viable to do by hand.


    (More recently).

    Current multiple playlists obfuscation drm on dvds (mostly Lionsgate), attempts to fool programs like anydvd/dvdfab into either choosing the wrong playlist(s) or choosing nothing (ie. fooling it into eliminating every single possible playlist).

    Since anydvd/dvdfab/makemkv are selling a one-click solution product, they have to put effort into defeating dvd/bluray drm as a general all-purpose algorithm which is applicable to many dvd/bluray discs. Case-by-case hacks are simply not good enough for a general purpose ripping product.
     
  5. DaveF

    DaveF Moderator
    Moderator

    Joined:
    Mar 4, 2001
    Messages:
    18,612
    Likes Received:
    1,754
    Trophy Points:
    9,110
    Location:
    One Loudoun, Ashburn, VA
    Real Name:
    David Fischer
    Thankfully it's not quite so difficult.

    AnyDVD rips the disc to ISO. My impression is it does have a database of individual processes for various discs, as needed. And it's upgraded regularly.

    MakeMKV simply extracts elements into MKV containers, with no attempt or pretense to find a playlist.

    There are various enthusiasts who do the hard work of figuring out what the main playlist is, when a disc is obfuscated. And there are techniques for identifying the playlist when it's not obvious.

    All that's to say, I've not had to individually sort through numerous 99+ playlists to figure out a movie. I generally spend more time identifying the commentaries and special features.
     
  6. jcroy

    jcroy Cinematographer

    Joined:
    Nov 28, 2011
    Messages:
    2,769
    Likes Received:
    771
    Trophy Points:
    4,110
    Real Name:
    jr
    I wouldn't be surprised at all, if somebody has already figured out a way of data snooping on a legitimate computer software bluray player in action (such as powerdvd) and extracting the exact playlist(s).

    In the scenario of going through the playlists systematically, there's always shortcuts such as figuring out which segment is the introduction at the beginning of a movie, and which segment is the final rolling end credits + logos. With that information, one can eliminate the playlists which don't have these two segments respectively as the "first" and "last" in a playlist. (ie. Basically "divide and conquer" until there's one or a few candidate playlists left).
     
  7. jcroy

    jcroy Cinematographer

    Joined:
    Nov 28, 2011
    Messages:
    2,769
    Likes Received:
    771
    Trophy Points:
    4,110
    Real Name:
    jr
    In the case of anydvd, I strongly suspect the "database" is just a stopgap measure. (Dunno about dvdfab).

    For somebody to spend so much time and effort at cracking bluray repeatedly, I strongly suspect the head designer/engineer of anydvd is a hacker type who takes pride in his/her work. If one is familiar with individuals with a "hacker/engineer" type personality profile, one good guess is that they're interested in finding a general algorithm/formula which can cover all possible known applicable cases.

    With every new update of BD+ and Screenpass drm on blurays, the designers/engineers of anydvd continue to tinker with their general algorithm so that the "database" of hardcoded statistics (such as valid playlists) is no longer required. (Other purposes for contacting the database server, could be for user/subscriber authentication type purposes and maybe some of the "general data" such as updated "processing keys").
     
  8. jcroy

    jcroy Cinematographer

    Joined:
    Nov 28, 2011
    Messages:
    2,769
    Likes Received:
    771
    Trophy Points:
    4,110
    Real Name:
    jr
    (On a more general tangent).

    Over the years I got the impression the makemkv folks were rather slow on the up and up, when it came to cracking newer drm schemes.

    For example in the case multiple playlists obfuscation drm on dvd (such as Disney and Paramount dvd titles from the late-2000s and early->mid 2010s), I get the impression they might be running a "brute force" method of going through every single playlist and eliminating the ones which don't look "sane". (Judging by the extra realtime messages dumped by makemkv in action on such dvd discs). Basically an automated computer coded version of what I would do systematically by hand via reading *.ifo files.
     
  9. jcroy

    jcroy Cinematographer

    Joined:
    Nov 28, 2011
    Messages:
    2,769
    Likes Received:
    771
    Trophy Points:
    4,110
    Real Name:
    jr
    (Some more rambling thoughts on multiple playlists obfuscation drm).

    In the scenario of makemkv not doing much of anything to choose "correct" playlists on blurays, it would suggest their programmers might not be too savvy about cracking the extra screenpass drm on bluray. Brute force type approaches of going through every playlist might not be effective anymore.

    Without any obvious hardcoded shortcuts (such as a database), I strongly suspect a semi-successful general algorithm would probably involve "tracing" through the bluray java code underlying the multiple playlists obfuscation. (In principle it is also possible to further "obfuscate/encrypt" the java code itself on disc and in memory, until it is de-obfuscated at the moment before execution. Tricks like this have been around for decades).


    (From an historical perspective).

    In contrast in the case of dvd's primitive version of multiple playlist obfuscation drm, the garbage playlists were so obvious even by just reading through the "program chains" in the *.ifo files. (This dvd playlist obfuscation drm construction also had an unrelated annoying dual purpose, which would fool the computer into thinking there were was over dozens gigabytes of data on the disc, when the actual real physical dvd disc itself only had less than 9 gigabytes capacity).

    Back in the day circa late-1990s and early-2000s, the few cases where I saw legitimate uses of multiple playlists for non-drm purposes, were typically dvds which had several multiple cuts of a movie on the same disc. For example, such as the two-disc Terminator2 dvd I had, where the different playlists were for different cuts of the movie. One playlist played Terminator2 with all the deleted scenes inserted back into the movie.

    By the time it was the late-2000s and afterwards, Disney and Paramount were kinda lazy and didn't bother doing many movies anymore with multiple versions on the same dvd disc. In the case of Paramount, they seemed to just release another dvd (or bluray) of a movie with a different cut (such as Bad Grandpa or Star Trek: Into Darkness) in an attempt to squeeze more cash from hardcore fans.

    Due to such laziness on the part of Disney and Paramount, the correct playlist could be found by going through every *.ifo file and finding the one with the cell vob ids in ascending numerical order (or almost like 1, 2, 3, 4, 5, 6, 7, 8, etc ... yada yada yada). This could be done easily by hand. (The garbage playlists had the cell vob ids in a wrong semi-random numerical sequence).

    Further refinements to fool then-current versions of dvd ripping programs in those days (ie. dvdfab, etc ...), modified this scrambling. For example, one sneaky refinement was having many garbage playlists which listed all the cell vob ids in ascending numerical order (ie. 1, 2, 3, 4, etc ....). But if one reads through the "program chains" corresponding to these garbage playlists, it turned out the addresses of the cells were deliberately incorrect. In the end, it turned out the correct playlist with the cell vob ids in ascending numerical order was the one which had all the correct cell addresses. (In *.ifo files, there is separate data table which lists all the addresses of the individual cells used by the playlists, and sometimes even some unused cells).


    (More recently).

    Since mid-late 2013, many Lionsgate dvd releases further refined this multiple playlist obfuscation drm. (Many of these titles I have were bluray/dvd/digital combo packs). Past Disney/Paramount versions of multiple playlist obfuscation drm typically only had one correct playlist and tons of garbage ones.

    How Lionsgate upped the ante, was having two or more correct playlists which were identical. Apparently during playback of these particular Lionsgate dvds, it would randomly choose to play one of these correct playlists. (There is explicit machine code in the videomanager's *.ifo file, which does this random choosing). I have watched many of these Lionsgate titles from the extracted dvd isos with all the basketcase drm left intact, and noticed that indeed it does choose randomly which correct playlist to play. (Playing a dvd iso on VLC, will tell you exactly which playlist is being played in realtime).

    For a program like makemkv, this Lionsgate multiple correct playlist issue shouldn't be a problem since makemkv doesn't bother preserving the dvd menus. (IIRC, makemkv will explicitly check whether a particular playlist is identical to another playlist and won't bother listing the duplicates).

    In contrast for dvdfab and anydvd + clonedvd, these programs appear to have an explicit interest in preserving the menus and removing all the basketcase drm such that the "cleaned up" *.ifo + *.vob files can be burned to a dvdr disc and played on a generic standalone dvd player without any problems. (ie. They are today's successor to previous ripandburn programs like dvdshrink).



    (More generally).

    How exactly dvdfab and anydvd figures out which dvd playlists are correct and which ones are garbage, is largely speculation. Since dvdfab says very little to nothing on their own official message board forums, it is very difficult to figure out what they're thinking and doing. I have been unable to figure out whether dvdfab's programmers are developing a general algorithm, or whether they are just kludging together a large number of case-by-case specific hacks. (Or some combination of the two).

    In contrast in the case of anydvd, the (alleged) developers/architects (or their official mouthpiece pr representatives) are somewhat more vocal on their own official anydvd message board forums. From reading through many old posts on their official forum, I get the impression the developers/architects are probably producing a finely tuned general algorithm. (ie. The hope is that a general algorithm does not "regress" and break and/or create new "false positives" on older previous dvd/bluray discs which were handled and settled with years ago. This would likely require some "fine tuning" every now and then).
     
  10. jcroy

    jcroy Cinematographer

    Joined:
    Nov 28, 2011
    Messages:
    2,769
    Likes Received:
    771
    Trophy Points:
    4,110
    Real Name:
    jr
    I was bored last night and decided to look into makemkv more, and guessing what primitive general algorithm they might be using on dvds with extra basketcase drm.

    In the case of the Disney/Paramount variety of multiple playlists obfuscation drm, it appears all they're doing is just choosing the playlist with the vob ids in ascending numerical order with the correct cell addresses. Basically what I would be doing by hand by skimming through the *.ifo files.


    In the case of many Lionsgate dvd titles since mid-2013, sometimes makemkv gets fooled into choosing two different playlists which both have the cells vob ids in ascending numerical order. On a closer examination, one of these playlists does not have any cells with deliberate bad sectors in them to begin with, while the second playlist does indeed contain "sick cells" with deliberate bad sectors without any cell commands to jump over these "sick cells". (Makemkv has an additional algorithm which strips out these "sick cells" by hand).

    Previously when I looked through the *.ifo files from these same Lionsgate dvds, it was obvious that the garbage playlists were the ones which either had the cell vob ids in a semi-random numerical order, and/or had "sick cells" which contained deliberate bad sectors with no cell commands to jump over these "sick cells". (In the case of dvdfab/anydvd, I strongly suspect their general algorithms will parse through the playlists to find which cells are jumped over and never played at all, but they do not assume that the playlist with the ascending cell vob ids in numerical order is necessarily the correct one).


    Recently I purchased three Lionsgate dvd/bluray combo pack titles which were released earlier this year and last year, and decided to take a closer look at the multiple playlist obfuscation on the dvd copies. (The titles were Deepwater Horizon, Hacksaw Ridge, and Divergent: Allegiant). It turns out for these recent titles, Lionsgate is no longer using deliberate bad sectors drm anymore, but they are still using multiple playlist obfuscation (along with the drm filesystem construction which fools the computer into thinking the disc has over 24+ gigabytes, while the actual real physical disc only has around 8.5 gigabytes). As far as I can tell, the easy "algorithmic" ways of eliminating garbage playlists are no longer effective on these particular Lionsgate dvd discs.

    To amuse myself briefly, I attempted to edit the *.ifo files in the easiest "obvious" manner to force the videomanager into only using one of the "sane" playlists. (Recall that the Lionsgate multiple playlist obfuscation drm had two or more correct identical playlists, which the videomanager randomly chose to play). Of all the easiest "obvious" hacks, none of them worked properly.

    Whoever at Sony/Lionsgate that designed this particular recent variant of the multiple playlist obfuscation (without any deliberate bad sectors drm), was very thorough and clever in their thinking.

    I looked through the anydvd message board archives, and noticed the head designers/architects initially took a long time to find a solution to their general algorithm for this particular recent Liongate variant of multiple playlist obfuscation. (Normally the anydvd designers only require a few days to solve problems with new drm variants/methods). Last year, IIRC one of the first then-new Lionsgate dvd titles which had this new variant of multiple playlist obfuscation without any deliberate bad sectors drm, was the Hunger Games: Mockingjay 2 dvd. (At the time, I didn't think much about analyzing the Mockingjay 2 dvd's basketcase drm further).

    In the end it took anydvd's designers/programmers around 5 months, to eventually find a solution for their general algorithm which could handle this recent drm variant on Mockingjay 2 and other subsequent Lionsgate releases over 2016. I'm sure Sony/Lionsgate are watching the anydvd folks closely, to see what new drm variants in multiple playlist obfuscation are difficult to remove (by trivial means).
     
  11. 31 Aug 17, 2017 at 12:36 PM
    Last edited: Aug 17, 2017 at 1:32 PM
    jcroy

    jcroy Cinematographer

    Joined:
    Nov 28, 2011
    Messages:
    2,769
    Likes Received:
    771
    Trophy Points:
    4,110
    Real Name:
    jr
    I was bored at a lunch today, and decided to search through the wikileaks archives (via googling) to see whether there are any leaked Sony documents which discussed dvd/bluray type drm. I won't bother with any links, but there is indeed several leaked Sony documents of this type.

    Everybody has long suspected the drm folks are closely watching the message board forums dedicated to programs like dvdfab, anydvd, etc .... (ie. Know your enemy). In these leaked Sony pp presentation documents, it indeed confirms that this is exactly what the drm folks are actively doing. There's even some screenshots from these forums in these presentation documents.
     
    DaveF likes this.
  12. 32 Aug 17, 2017 at 12:51 PM
    Last edited: Aug 17, 2017 at 1:05 PM
    jcroy

    jcroy Cinematographer

    Joined:
    Nov 28, 2011
    Messages:
    2,769
    Likes Received:
    771
    Trophy Points:
    4,110
    Real Name:
    jr
    (On a more general big picture perspective).

    Nowadays the main targets of the MPAA and the movie companies, are most likely the parent companies of anydvd, dvdfab, and makemkv.

    I strongly suspect the primary reason the MPAA wants to destroy anydvd, dvdfab, makemkv, etc ... is because the latter companies are making money that the MPAA and movie companies don't ever see a cent from, and that they are based in jurisdictions which the american and european "strong arms of the law" have no direct control over (such as China, etc ...).

    Currently the MPAA and movie companies don't seem to waste their time and political capital on destroying the few open source programs/libraries which crack dvd/bluray drm. I suspect by the time all the dvd patents expire over the next decade or so, the MPAA simply won't care anymore about dvd drm.
     
  13. DaveF

    DaveF Moderator
    Moderator

    Joined:
    Mar 4, 2001
    Messages:
    18,612
    Likes Received:
    1,754
    Trophy Points:
    9,110
    Location:
    One Loudoun, Ashburn, VA
    Real Name:
    David Fischer
    What I have wanted is affordable and reliable solutions for home media libraries. I know there are pirates looking for free movies. But I want an elegant and technically sophisticated way to put all my movies on hard drives and get the physical media into storage instead of being living room "decorations".

    I buy HT gear. I buy computers.I'd have spent money on a good "hollywood" solution (I looked at Kaleidoscape but it's a stupidly-expensive kludge.) This is becoming moot with streaming. But in the interim, studios and hardware makers have left money on the table with their obstinance.
     
  14. jcroy

    jcroy Cinematographer

    Joined:
    Nov 28, 2011
    Messages:
    2,769
    Likes Received:
    771
    Trophy Points:
    4,110
    Real Name:
    jr
    Definitely.

    I'll probably eventually move on to streaming completely, once it becomes ubiquitous and inexpensive.

    Currently I don't really have much motivation anymore to buy recent movies on bluray (or dvd), such as Marvel stuff and various action, scifi, etc ... type stuff. Nowadays I end up mostly watching Marvel stuff when they end up on a basic cable channel I am subscribed to, albeit with a 2 or 3 or 4 year "delay". Most other current movies of interest to me (such as scifi, fantasy, action, etc ...), eventually end up on basic cable channels within 2 years or so.

    I find that I'm not distracted at all by channel logos, commercials, etc ... when I'm watching a generic superhero movie on basic cable.
     
  15. jcroy

    jcroy Cinematographer

    Joined:
    Nov 28, 2011
    Messages:
    2,769
    Likes Received:
    771
    Trophy Points:
    4,110
    Real Name:
    jr
    (This is strictly anecdotal and semi-historical).

    Most of the hardcore pirates I knew of back in the day (mostly former classmates from college), were the types who just collected tons of cracked software, movies, etc ... for bragging rights. They rarely ever used any of the cracked software, movies, etc ... they had accumulated.

    By the time Netflix dvd-by-mail flat-rate type services were common around a decade ago or so, pirating movies was no longer of interest to many such pirates. (ie. It was no longer something to brag about, unless it was a movie that wasn't released yet). The wannabe pirate types who were still into accumulating tons of movies, typically were the ones who were the "fast turnaround" types on dvd-by-mail type services like Netflix. Typically they would rip an iso copy of the movie to the computer, and send the disc back within hours. (Some were even sneaky and had two or more accounts under different names, such as their wife's name, etc ...).

    Nowadays it seems like in the pecking order of pirates, movies ripped from dvd/bluray discs are largely a joke now. Something to brag about among their circles in regard to movies/shows, would be getting stuff that hasn't been released yet (such as from "screener" copies) and/or cracking 4Kbluray before anydvd does it.
     

Share This Page