Hey DM, a question on the dupe checking.

I've been having a discussion with James Coyle over messages being changed in transit that have Soft-CRs in them.

He made this comment:

It shouldn't matter if the characters are there or not, if your tosser is properly ignoring them as per the FTS then the dupe CRC value will be the > same whether they are there or not (as it is with Mystic). If your tosser
doesn't do that, then its not ignoring those characters its including them.

Does SBBSecho ignore Soft-CRs when doing dupe checking? If it does, is that a configurable item?

The reason I'm asking, is I have been getting dupes from a legacy system that appears to send out messages with them. (One from an SBBS hub, the other from a Mystic Hub.)
...deon


... It was completely quiet in the stadium - but noisy.

---
þ Synchronet þ Alterant | an SBBS in Docker on Pi!

Re: Dupe Checking...
By: alterego to Digital Man on Mon Mar 16 2020 21:44:08


alterego> I've been having a discussion with James Coyle over messages
alterego> being changed in transit that have Soft-CRs in them.

alterego> He made this comment:

alterego>> It shouldn't matter if the characters are there or not, if
alterego>> your tosser is properly ignoring them as per the FTS then
alterego>> the dupe CRC value will be the > same whether they are
alterego>> there or not (as it is with Mystic). If your tosser doesn't
alterego>> do that, then its not ignoring those characters its including
alterego>> them.

IMHO, he's using the wrong form of "ignore"... the proper form is to not do anything special because of them... it doesn't mean they are to be skipped during calculations or while processing the messages... if they are in the input, they should be in the output...


)\/(ark

---
þ Synchronet þ The SouthEast Star Mail HUB - SESTAR

Re: Dupe Checking...
By: Rampage to alterego on Mon Mar 16 2020 07:45 am

IMHO, he's using the wrong form of "ignore"... the proper form is to not do anything special because of them... it doesn't mean they are to be skipped during
calculations or while processing the messages... if they are in the input, they should be in the output...

Yeah, that is what I was thinking as well.
...deon


... Organization is the enemy of improvisation.

---
þ Synchronet þ Alterant | an SBBS in Docker on Pi!

Re: Dupe Checking...
By: alterego to Rampage on Mon Mar 16 2020 23:49:01

IMHO, he's using the wrong form of "ignore"... the proper form is to not
do anything special because of them... it doesn't mean they are to be
skipped during calculations or while processing the messages... if they
are in the input, they should be in the output...

alterego> Yeah, that is what I was thinking as well.

it falls hand in hand with the "do not modify mail in transit other than as necessary" statements... those modifications are generally

for echomail:
adjusting the path lines
adjusting the seenby lines
changing the destination address in the header

for netmail:
adding a VIA line

and that's it off the top of my head without digging out actual processing code and looking closer...


)\/(ark

---
þ Synchronet þ The SouthEast Star Mail HUB - SESTAR

Re: Dupe Checking...
By: alterego to Digital Man on Mon Mar 16 2020 09:44 pm

Hey DM, a question on the dupe checking.

I've been having a discussion with James Coyle over messages being changed in transit that have Soft-CRs in them.

He made this comment:

It shouldn't matter if the characters are there or not, if your tosser is properly ignoring them as per the FTS then the dupe CRC value will be the

same whether they are there or not (as it is with Mystic). If your

tosser doesn't do that, then its not ignoring those characters its including them.

Does SBBSecho ignore Soft-CRs when doing dupe checking?

If you have "StripSoftCRs" enabled in sbbsecho.ini, yes.

If it does, is that a configurable item?

Yup.

The reason I'm asking, is I have been getting dupes from a legacy system that appears to send out messages with them. (One from an SBBS hub, the other from a Mystic Hub.)

SBBSecho also checks for duplicate message-IDs, which should not be affected by any change to the message body text in transit.

digital man

This Is Spinal Tap quote #7:
Nigel Tufnel: That's just nitpicking, isn't it?
Norco, CA WX: 58.8øF, 62.0% humidity, 4 mph NW wind, 0.00 inches rain/24hrs

---
þ Synchronet þ Vertrauen þ Home of Synchronet þ [vert/cvs/bbs].synchro.net

Re: Dupe Checking...
By: Digital Man to alterego on Mon Mar 16 2020 02:36 pm

SBBSecho also checks for duplicate message-IDs, which should not be affected by any change to the message body text in transit.

I'm curious on this.

So two message-ID's with different content should still be dupe detected, with the second one being discarded? If this is true, then I dont think it is working, since I am seeing duplicate messages (same ID), but slightly different content (one with the soft-CRs, the other without).

I'll find some debugging if I have this correct?
...deon


... The older you get, the more important is is not to act your age.

---
þ Synchronet þ Alterant | an SBBS in Docker on Pi!

Re: Dupe Checking...
By: alterego to Digital Man on Tue Mar 17 2020 07:37 pm

with the second one being discarded? If this is true, then I dont think it is working, since I am seeing duplicate messages (same ID), but slightly different content (one with the soft-CRs, the other without).

OK, I looked, I dont think those messages have a MSGID kludge - so that would explain why they didnt get caught by dupe checking...

(I was confused by the hex code next to the date - it was the same for both messages, and I thought for a moment that was from the MSGID.)

All good :)
...deon


... When I was crossing the border into Canada, they asked if I had any firearms with me. I said, "What do you need?"

---
þ Synchronet þ Alterant | an SBBS in Docker on Pi!

Re: Dupe Checking...
By: alterego to Digital Man on Tue Mar 17 2020 07:37 pm

Re: Dupe Checking...
By: Digital Man to alterego on Mon Mar 16 2020 02:36 pm

SBBSecho also checks for duplicate message-IDs, which should not be affected by any change to the message body text in transit.

I'm curious on this.

So two message-ID's with different content should still be dupe detected, with the second one being discarded?

Correct.

If this is true, then I dont think it
is working, since I am seeing duplicate messages (same ID), but slightly different content (one with the soft-CRs, the other without).

I'll find some debugging if I have this correct?

It's working here. The log lines in sbbsecho.log look like this:
03/17/20 10:20:21 ERROR 1 (smb_addmsg duplicate X-FTN-MSGID: 1:342/200@fidonet 25b53e32 found in message #769) line 3615 adding message to wildcat!

digital man

Synchronet/BBS Terminology Definition #53:
PETSCII = PET Standard Code of Information Interchange (a.k.a. CBM ASCII) Norco, CA WX: 52.2øF, 60.0% humidity, 0 mph ESE wind, 0.32 inches rain/24hrs

---
þ Synchronet þ Vertrauen þ Home of Synchronet þ [vert/cvs/bbs].synchro.net