• Dupe Checking...

    From alterego@VERT/ALTERANT to Digital Man on Monday, March 16, 2020 21:44:08
    Hey DM, a question on the dupe checking.

    I've been having a discussion with James Coyle over messages being changed in transit that have Soft-CRs in them.

    He made this comment:

    It shouldn't matter if the characters are there or not, if your tosser is properly ignoring them as per the FTS then the dupe CRC value will be the > same whether they are there or not (as it is with Mystic). If your tosser
    doesn't do that, then its not ignoring those characters its including them.

    Does SBBSecho ignore Soft-CRs when doing dupe checking? If it does, is that a configurable item?

    The reason I'm asking, is I have been getting dupes from a legacy system that appears to send out messages with them. (One from an SBBS hub, the other from a Mystic Hub.)
    ...deon


    ... It was completely quiet in the stadium - but noisy.

    ---
    Synchronet Alterant | an SBBS in Docker on Pi!
  • From Rampage@VERT/SESTAR to alterego on Monday, March 16, 2020 07:45:26
    Re: Dupe Checking...
    By: alterego to Digital Man on Mon Mar 16 2020 21:44:08


    alterego> I've been having a discussion with James Coyle over messages
    alterego> being changed in transit that have Soft-CRs in them.

    alterego> He made this comment:

    alterego>> It shouldn't matter if the characters are there or not, if
    alterego>> your tosser is properly ignoring them as per the FTS then
    alterego>> the dupe CRC value will be the > same whether they are
    alterego>> there or not (as it is with Mystic). If your tosser doesn't
    alterego>> do that, then its not ignoring those characters its including
    alterego>> them.

    IMHO, he's using the wrong form of "ignore"... the proper form is to not do anything special because of them... it doesn't mean they are to be skipped during calculations or while processing the messages... if they are in the input, they should be in the output...


    )\/(ark

    ---
    Synchronet The SouthEast Star Mail HUB - SESTAR
  • From alterego@VERT/ALTERANT to Rampage on Monday, March 16, 2020 23:49:01
    Re: Dupe Checking...
    By: Rampage to alterego on Mon Mar 16 2020 07:45 am

    IMHO, he's using the wrong form of "ignore"... the proper form is to not do anything special because of them... it doesn't mean they are to be skipped during
    calculations or while processing the messages... if they are in the input, they should be in the output...

    Yeah, that is what I was thinking as well.
    ...deon


    ... Organization is the enemy of improvisation.

    ---
    Synchronet Alterant | an SBBS in Docker on Pi!
  • From Rampage@VERT/SESTAR to alterego on Monday, March 16, 2020 15:12:16
    Re: Dupe Checking...
    By: alterego to Rampage on Mon Mar 16 2020 23:49:01


    IMHO, he's using the wrong form of "ignore"... the proper form is to not
    do anything special because of them... it doesn't mean they are to be
    skipped during calculations or while processing the messages... if they
    are in the input, they should be in the output...

    alterego> Yeah, that is what I was thinking as well.

    it falls hand in hand with the "do not modify mail in transit other than as necessary" statements... those modifications are generally

    for echomail:
    adjusting the path lines
    adjusting the seenby lines
    changing the destination address in the header

    for netmail:
    adding a VIA line

    and that's it off the top of my head without digging out actual processing code and looking closer...


    )\/(ark

    ---
    Synchronet The SouthEast Star Mail HUB - SESTAR
  • From Digital Man@VERT to alterego on Monday, March 16, 2020 14:36:48
    Re: Dupe Checking...
    By: alterego to Digital Man on Mon Mar 16 2020 09:44 pm

    Hey DM, a question on the dupe checking.

    I've been having a discussion with James Coyle over messages being changed in transit that have Soft-CRs in them.

    He made this comment:

    It shouldn't matter if the characters are there or not, if your tosser is properly ignoring them as per the FTS then the dupe CRC value will be the
    same whether they are there or not (as it is with Mystic). If your
    tosser doesn't do that, then its not ignoring those characters its including them.

    Does SBBSecho ignore Soft-CRs when doing dupe checking?

    If you have "StripSoftCRs" enabled in sbbsecho.ini, yes.

    If it does, is that a configurable item?

    Yup.

    The reason I'm asking, is I have been getting dupes from a legacy system that appears to send out messages with them. (One from an SBBS hub, the other from a Mystic Hub.)

    SBBSecho also checks for duplicate message-IDs, which should not be affected by any change to the message body text in transit.

    digital man

    This Is Spinal Tap quote #7:
    Nigel Tufnel: That's just nitpicking, isn't it?
    Norco, CA WX: 58.8F, 62.0% humidity, 4 mph NW wind, 0.00 inches rain/24hrs

    ---
    Synchronet Vertrauen Home of Synchronet [vert/cvs/bbs].synchro.net
  • From alterego@VERT/ALTERANT to Digital Man on Tuesday, March 17, 2020 19:37:14
    Re: Dupe Checking...
    By: Digital Man to alterego on Mon Mar 16 2020 02:36 pm

    SBBSecho also checks for duplicate message-IDs, which should not be affected by any change to the message body text in transit.

    I'm curious on this.

    So two message-ID's with different content should still be dupe detected, with the second one being discarded? If this is true, then I dont think it is working, since I am seeing duplicate messages (same ID), but slightly different content (one with the soft-CRs, the other without).

    I'll find some debugging if I have this correct?
    ...deon


    ... The older you get, the more important is is not to act your age.

    ---
    Synchronet Alterant | an SBBS in Docker on Pi!
  • From alterego@VERT/ALTERANT to Digital Man on Tuesday, March 17, 2020 21:39:34
    Re: Dupe Checking...
    By: alterego to Digital Man on Tue Mar 17 2020 07:37 pm

    with the second one being discarded? If this is true, then I dont think it is working, since I am seeing duplicate messages (same ID), but slightly different content (one with the soft-CRs, the other without).

    OK, I looked, I dont think those messages have a MSGID kludge - so that would explain why they didnt get caught by dupe checking...

    (I was confused by the hex code next to the date - it was the same for both messages, and I thought for a moment that was from the MSGID.)

    All good :)
    ...deon


    ... When I was crossing the border into Canada, they asked if I had any firearms with me. I said, "What do you need?"

    ---
    Synchronet Alterant | an SBBS in Docker on Pi!
  • From Digital Man@VERT to alterego on Tuesday, March 17, 2020 10:37:04
    Re: Dupe Checking...
    By: alterego to Digital Man on Tue Mar 17 2020 07:37 pm

    Re: Dupe Checking...
    By: Digital Man to alterego on Mon Mar 16 2020 02:36 pm

    SBBSecho also checks for duplicate message-IDs, which should not be affected by any change to the message body text in transit.

    I'm curious on this.

    So two message-ID's with different content should still be dupe detected, with the second one being discarded?

    Correct.

    If this is true, then I dont think it
    is working, since I am seeing duplicate messages (same ID), but slightly different content (one with the soft-CRs, the other without).

    I'll find some debugging if I have this correct?

    It's working here. The log lines in sbbsecho.log look like this:
    03/17/20 10:20:21 ERROR 1 (smb_addmsg duplicate X-FTN-MSGID: 1:342/200@fidonet 25b53e32 found in message #769) line 3615 adding message to wildcat!

    digital man

    Synchronet/BBS Terminology Definition #53:
    PETSCII = PET Standard Code of Information Interchange (a.k.a. CBM ASCII) Norco, CA WX: 52.2F, 60.0% humidity, 0 mph ESE wind, 0.32 inches rain/24hrs

    ---
    Synchronet Vertrauen Home of Synchronet [vert/cvs/bbs].synchro.net