Is there a way to get ADDFILES to only update the upload date on files that have a newer timestamp than before?

Basically - I'd like to be able to mirror some files, but make sure that the newest version of files is listed as recently uploaded, without updating the upload date of any files that weren't updated (if that makes sense.)

---
þ Synchronet þ Deep Space '94 - deepspace94.com - The Best 1994 Had to Offer

Re: ADDFILES and upload date?
By: Deepthaw to All on Thu Jul 06 2017 09:10 pm

Is there a way to get ADDFILES to only update the upload date on files that have a newer timestamp than before?

Basically - I'd like to be able to mirror some files, but make sure that the newest version of files is listed as recently uploaded, without updating the upload date of any files that weren't updated (if that makes sense.)

There was no existing feature to do that, but I just added it for ya. Use the '-p' option (in this morning's build) and addfiles will compare the file's timestamp with the upload timestamp and skip files that haven't been changed since.

digital man

Synchronet/BBS Terminology Definition #14:
DOCSIS = Data Over Cable Service Interface Specification
Norco, CA WX: 75.3øF, 56.0% humidity, 0 mph WNW wind, 0.00 inches rain/24hrs

---
þ Synchronet þ Vertrauen þ Home of Synchronet þ telnet://vert.synchro.net

Re: ADDFILES and upload date
By: Digital Man to Deepthaw on Fri Jul 07 2017 12:44 am

There was no existing feature to do that, but I just added it for ya. Use the '-p' option (in this morning's build) and addfiles will compare the file's timestamp with the upload timestamp and skip files that haven't been changed since.

Wow, thanks! The next question is - do you mind if I mirror the Synchronet folder from your site onto my own, including the nightly builds?

I figure wget can already be configured to pull down only those files that have changed, and this change to ADDFILES will make it so it doesn't appear to my users that the entire directory has been uploaded anew every night.

---
þ Synchronet þ Deep Space '94 - deepspace94.com - The Best 1994 Had to Offer

Re: ADDFILES and upload date
By: Deepthaw to Digital Man on Fri Jul 07 2017 08:01 am

Re: ADDFILES and upload date
By: Digital Man to Deepthaw on Fri Jul 07 2017 12:44 am

There was no existing feature to do that, but I just added it for ya. Use the '-p' option (in this morning's build) and addfiles will compare the file's timestamp with the upload timestamp and skip files that haven't been changed since.

Wow, thanks! The next question is - do you mind if I mirror the Synchronet folder from your site onto my own, including the nightly builds?

Sure, that's fine.

I figure wget can already be configured to pull down only those files that have changed, and this change to ADDFILES will make it so it doesn't appear to my users that the entire directory has been uploaded anew every night.

Let me know how the addfiles -p enhancement works for ya.

digital man

Synchronet/BBS Terminology Definition #21:
FF = Form Feed (ASCII 12, Ctrl-L)
Norco, CA WX: 98.3øF, 30.0% humidity, 5 mph ESE wind, 0.00 inches rain/24hrs

---
þ Synchronet þ Vertrauen þ Home of Synchronet þ telnet://vert.synchro.net

Re: ADDFILES and upload date
By: Digital Man to Deepthaw on Fri Jul 07 2017 12:24 pm

Let me know how the addfiles -p enhancement works for ya.

I'll run my script around 10am tomorrow and see how it works out. with any luck, the only things that'll show up as new will be the nightly builds.

---
þ Synchronet þ Deep Space '94 - deepspace94.com - The Best 1994 Had to Offer

Re: ADDFILES and upload date
By: Digital Man to Deepthaw on Fri Jul 07 2017 12:24 pm

Let me know how the addfiles -p enhancement works for ya.

I gave it a few test runs, and it seems to be working with one hang-up : the date verification doesn't seem to be working for the first file the directory. It always thinks it's new and adds it. I tested it by deleting the file it kept adding on every run (despite wget not updating it) and it would instead add the next file in the directory.

Windows Server 2003 32-bit (I noticed some #ifdef _win32 directives in the source, in case that matters.)

---
þ Synchronet þ Deep Space '94 - deepspace94.com - The Best 1994 Had to Offer

Re: ADDFILES and upload date
By: Deepthaw to Digital Man on Sat Jul 08 2017 02:23 am

Re: ADDFILES and upload date
By: Digital Man to Deepthaw on Fri Jul 07 2017 12:24 pm

Let me know how the addfiles -p enhancement works for ya.

I gave it a few test runs, and it seems to be working with one hang-up : the date verification doesn't seem to be working for the first file the directory. It always thinks it's new and adds it. I tested it by deleting the file it kept adding on every run (despite wget not updating it) and it would instead add the next file in the directory.

Interesting. I don't really have an explanation for that one. What is the upload date/time for that file (that's the date that is compared)?

digital man

Synchronet/BBS Terminology Definition #4:
BBS = Bulletin Board System
Norco, CA WX: 98.8øF, 31.0% humidity, 8 mph ESE wind, 0.00 inches rain/24hrs

---
þ Synchronet þ Vertrauen þ Home of Synchronet þ telnet://vert.synchro.net

Re: ADDFILES and upload date
By: Digital Man to Deepthaw on Sat Jul 08 2017 01:00 pm

Interesting. I don't really have an explanation for that one. What is the upload date/time for that file (that's the date that is compared)?

When I just ran it, the file in question (dokuauth.zip) had an upload timestamp of 07/08/17, 2:22PM. After rerunning ADDFILES, it was marked as new and as having an upload timestamp of 07/08/17, 4:45PM.

So I edited my batch file, and told it to delete dokuauth.zip before running ADDFILES. Now the new first file in the directory, sbbs313a.zip is marked as being new despite wget not updating it in any way. So I edited the script to delete *that* file before running ADDFILES, and now sbbs314a.zip is the one marked as new every time I run it.

It seems to be working fine for every file except the very first one in the directory.

Windows Server 2003 R2, 32-bit. Here's the contents of the batch file:

D:
CD \BBS
c:\utils\wget --tries=5 --append-output=C:\SBBS\mirror.log --ftp-user=anonymous --ftp-password=deepthaw@deepspace94.com -N -r -l inf ftp://ftp.synchro.net
del D:\BBS\ftp.synchro.net\files.bbs
del D:\BBS\ftp.synchro.net\file_id.diz
c:\sbbs\exec\addfiles MAINSBBS -p -z -s -f

The wget log I'm saving indicates it is working as expected: it downloads FILE_ID.DIZ and FILES.BBS every time (because I keep deleting them) but skips everything else that is unchanged vs what I have on disk.

Every time ADDFILES completes, whatever is the first result of a directory listing gets an upload date of when the batch was ran, but everything else works as expected (only the nightly builds got their upload date changed, which was the intended behavior.)

---
þ Synchronet þ Deep Space '94 - deepspace94.com - The Best 1994 Had to Offer

Re: ADDFILES and upload date
By: Deepthaw to Digital Man on Sat Jul 08 2017 04:57 pm

Every time ADDFILES completes, whatever is the first result of a directory listing gets an upload date of when the batch was ran, but everything else works as expected (only the nightly builds got their upload date changed, which was the intended behavior.)

Thanks for the detailed report. I think I've fixed the problem now in CVS.

digital man

Synchronet/BBS Terminology Definition #6:
BinkP = BinkD Protocol
Norco, CA WX: 90.0øF, 41.0% humidity, 7 mph E wind, 0.00 inches rain/24hrs

---
þ Synchronet þ Vertrauen þ Home of Synchronet þ telnet://vert.synchro.net