[NZBGET] Unpack script modifications

[lordy]

Updated again Version 20080902-01 : Some bug fixes and some features..

It's expanding like the blob !
-------------

All,

Here are some mods I made to the unpak script.
The main features are:

1. Rar based. Will unpack archives without par files.
2. Only delete rar volumes whose contents were successfully extracted
3. A few more quoted variables to cope with spaces in filenames etc.

Download the script from URL below..

http://www.prodynamic.co.uk/nzbget/nzbget.html

[werner]

nice work great to see somenice scripters here

[hugbug]

Great job, ydrol.
I like how structured it looks now
I'll take that version for nzbget distrubition.

ydrol, don't you want to add your name to copyright list?
(I guess the copryright notice will be soon larger than the code part )

[lordy]

(08-27-2008 10:08 PM)hugbug Wrote:  Great job, ydrol.
I like how structured it looks now
I'll take that version for nzbget distrubition.

Hold up! Spotted a slight buggette - it will try to unpack damaged rars 3 times - I'll be fixing it shortly. Look for v080827-2 any minute now..

Quote:ydrol, don't you want to add your name to copyright list?
(I guess the copryright notice will be soon larger than the code part )

Yeah I'll slap it on there

[hugbug]

That's cool to unrar files even if there were no par-files.

I would add an extra check for existence of "_brokenlog.txt" before trying to unrar (exit with a warning in this case).

The line should be printred before ?

[Philos]

WOW!
Very nice, lots of options and really structured.
I'll be using this one from now on.
Thanks

[hugbug]

(08-28-2008 08:37 AM)hugbug Wrote:  I would add an extra check for existence of "_brokenlog.txt" before trying to unrar (exit with a warning in this case).

if it wasn't checked/repaired, of cause

[lordy]

Thanks for the feedback, from inspection I think there is another buggette
I think if it fails to unpack a rar file with format *.part01.rar then it will try each part in turn.
Look out for a new fix

[lordy]

Ive Updated script. to v 20080828-1 Tested as much as I could (nothing too exciting to download at the moment )
Please test away and report any 'undocumented features'

Cheers!

[Philos]

Yes found something.

The script doesnt unrar double rarred files, rar set inside a rar set.
This was a request of the German forum and Werner.
Some German posts are rar in rar and the original script will unrar and after that try to unrar again.

I tested this with pch unrar test, its on usenet and posted 38 days ago.
Its an img file (nothing playable) inside rar inside rar...
http://binsearch.info/?q=pch+unrar&max=2...50&server=

[hugbug]

It still does not check for "_brokenlog.txt". Why so? That file is created during download if a CRC error occured or an article could not be fetched, not during parcheck. If you don't want to check always for the file, may be you could make an option to enable that extra check? It should be very handy for PCH (slow CPU) to save 30-40 minutes of unraring, which will fail anyways.

About check of file sizes. If the option "DirectWrite" is enabled (I recommend to enable it on Ext2/Ext3 partions) the size of created files is always correct, even if some articles were missing.

[lordy]

@Philos New one uploaded (see 1st post)

---

@hugbug

(08-28-2008 09:18 PM)hugbug Wrote:  It still does not check for "_brokenlog.txt". Why so? That file is created during download if a CRC error occured or an article could not be fetched, not during parcheck. If you don't want to check always for the file, may be you could make an option to enable that extra check? It should be very handy for PCH (slow CPU) to save 30-40 minutes of unraring, which will fail anyways.

Hi, it's on the roadmap I just didn't have a sample _brokenlog.txt so I could properly differentiate between a repaired file and one that is just broken. I haven't yet run through all the scenarios. Which I think are:

1 - Rar ok - no par -> no brokenlog
2 - Rar CRC damaged - no par -> _brokenlog present with just affected rar volumes listed.
3 - Rar missing parts from nzb - no par -> no brokenlog.

4 - Rar ok - par file -> no brokenlog
5 - Rar CRC damaged or missing parts - par fixed -> _brokenlog present with affected rar volumes followed by success message based on Par
6 - Rar CRC damaged or missing parts - par failed -> $parFail set (script will exit), _brokenlog present with affected rar volumes then Par failure message.

I will write code to differentiate between 2 and 5.
Because I want the script to cope with multiple rars per NZB I may have to code the mapping to get rar volumes associated with par files?
It also means I have to look inside the _brokenlog.txt?

I did this for the nzbget-housekeeper , by copying the par file to an empty folder and doing parverify on it, and parsing the output. But not tonight Just need to make sure I'm not over-engineering it

Also I try to avoid the code being dependent on nzbget behaviour too much, just in case a log format changes here or there, but if there is a yenc CRC error then I should take advantage of the _brokenlog.txt for sure!

Quote:About check of file sizes. If the option "DirectWrite" is enabled (I recommend to enable it on Ext2/Ext3 partions) the size of created files is always correct, even if some articles were missing.

Ah good point.

Also what happens to brokenlog.txt when nzb is added to the queue with same name as previous one. I sometimes do this if adding in more par files.
PS Any further thoughts appreciated.

[hugbug]

I would make it simpler and do not analyze content of _brokenlog.txt:
1) if parameter "indication of failed par-jobs for current nzb-file" indicates failed jobs, then do nothing and exit from script;
2) if parameter "result of par-check" is set to "checked and sucessfully repaired", then ignore _brokenlog.txt and do processing;
3) if parameter "result of par-check" is set to "not checked: par-check disabled or nzb-file does not contain any par-files" then check if _brokenlog.txt exists. If it exists - exit without processing. If not - try to unrar.
4) other values for "result of par-check" ("checked and failed to repair" or "checked and can be repaired but repair is disabled") cause immediate exit.

I would not try to determine what collection is damaged. That would be to much intelligence

[lordy]

Done (see 1st post)

I wont add it now but I just realised its only worth doing the additional sanity checks if ParCheck is 0

[hugbug]

It is really big now

Small suggestions to make the script easier portable on other systems:
1) Remove line "export PATH=$PATH:/mnt/syb8634/bin";
2) In function "NZBGET()" make the command that calls nzbget to be a setting (with full path to nzbget);
3) Make a setting for unrar-command (path to unrar).