Hiya DM...
I originally posted this in Sysops but there are a few on Fido that are also interested in the answer/resolution.
Hiya DM...
Quick question regardig sbbsecho and dupe checking. Does it check for X-FTN-MSGID automatically without turning on the dupe check in SCFG?
On 04-20-15 19:09, Digital Man wrote to Bill McGarrity <=-
Re: Dupes...
By: Bill McGarrity to Digital Man on Sun Apr 19 2015 11:39 am
Hiya DM...
I originally posted this in Sysops but there are a few on Fido that are also interested in the answer/resolution.
Hiya DM...
Quick question regardig sbbsecho and dupe checking. Does it check for X-FTN-MSGID automatically without turning on the dupe check in SCFG?
Yes, duplicate message-IDs are always checked, even if duplicate
message body text checking is disabled (the only exception is the "bad echo" area where duplicate message-IDs are allowed).
OK, reply from Rob on the issues of dupe checking. He is speaking
about Joe's system as that is who Wilfred was originally talking
about.
If the seen-bys were in-fact stripped, then when Joe's system received
the message a second time, it would not know it's a duplicate until it
performed a duplicate message check. If the message was indeed a
duplicate, then it would not be imported into Joe's system *or*
forwarded to his links.
So this takes care of one of the issues that was discussed. Before a message gets forwarded it is checked.
Joe,
SBBSecho only checks for duplicate messages when importing into a
local message base. If you're a hub for an echo, any messages you
receive from one link (up or downlink) will be propagated to the
others without any dupe checking.
-Rob
Wilfred,
The duplicate message checking occurs in the Synchronet Message Base
(SMB) library, which is not specific to FidoNet or any other
networking technology. So whether a message is imported via QWK, NNTP, SMTP, PostLink, FTN, or whatever- future-networking-technology, it is subjected to the same database of hashes to detect duplicates.
When an FTN message packet is processed by SBBSecho but the message is
not imported into a local message base (e.g. pass-through area or
passing between up/downlinks), the SMB library is thus not involved
(the message would go straight from one FTN packet to another FTN
packet), never touching a Synchronet message base.
-Rob
Instead, what I think is happening, is the seen-bys were *not*
stripped, so Joe's system recgonizes this as a circular path message
(due to his address already existing in the SEEN-BYs), does not bother
with any dupe checking, and then forwards the message to his links.
This could be an issue everyone seems to worried about.
Joe could easily change this behavior by disabling the Circulat Path
checking in sbbsecho (adding NOPATHCHECK to his sbbsecho.cfg or
enabling the equivalent option in EchoCfg) - this would require no
change to SBBSecho.
One fix....
Alternatively, Joe could leave circular-path detection enabled (which
I think is a good thing) and I could add an option to SBBSecho to
*not* forward circular messages to links. This would not require dupe
checking at all.
This is the fix I asked Rob to do.
theSo to recap: SBBSecho's dupe checking works fine and indeed duplicate
messages are detected *before* sent to links (and dupe messages are
not forwarded). However, a detected circular path prevents any
duplicate message checking and this is the area for potential change
in behavior.
So there is the explaination. Hopefully within a few weeks the new option can be added to SBBScho so all bases are covered. Right now I've added
NOCHECKPATH to see if that at least helps. Just know IF duplicate MSGID are detected, they are stripped before being sent to links.
* Originally in FIDONEWS
* Crossposted in SYNCHRONET
Hi Bill,
On 2015-04-23 22:44:00, you wrote to All:
OK, reply from Rob on the issues of dupe checking. He is speaking about Joe's system as that is who Wilfred was originally talking
about.
If the seen-bys were in-fact stripped, then when Joe's system received
the message a second time, it would not know it's a duplicate until it
performed a duplicate message check. If the message was indeed a
duplicate, then it would not be imported into Joe's system *or*
forwarded to his links.
So this takes care of one of the issues that was discussed. Before a message gets forwarded it is checked.
That is not what Rob told me and Joe last year:
On 19-Jan-14 22:58, Rob Swindell wrote:
Joe,
SBBSecho only checks for duplicate messages when importing into a
local message base. If you're a hub for an echo, any messages you receive from one link (up or downlink) will be propagated to the
others without any dupe checking.
On Tue, Jan 21, 2014 at 10:08 PM, Rob Swindell <rob@synchro.net> wrote:
Wilfred,
The duplicate message checking occurs in the Synchronet Message Base (SMB) library, which is not specific to FidoNet or any other
networking technology. So whether a message is imported via QWK, NNTP, SMTP, PostLink, FTN, or whatever- future-networking-technology, it is subjected to the same database of hashes to detect duplicates.
When an FTN message packet is processed by SBBSecho but the message is not imported into a local message base (e.g. pass-through area or passing between up/downlinks), the SMB library is thus not involved (the message would go straight from one FTN packet to another FTN packet), never touching a Synchronet message base.
-Rob
So I think you miss understood what Rob is saying in his reply to you?
stripped, so Joe's system recgonizes this as a circular path message
(due to his address already existing in the SEEN-BYs), does not bother
with any dupe checking, and then forwards the message to his links.
This could be an issue everyone seems to worried about.
I would call it a bug. But Rob fixed it with the new option...
Joe could easily change this behavior by disabling the Circulat Path
checking in sbbsecho (adding NOPATHCHECK to his sbbsecho.cfg or
enabling the equivalent option in EchoCfg) - this would require no
change to SBBSecho.
One fix....
Alternatively, Joe could leave circular-path detection enabled (which
I think is a good thing) and I could add an option to SBBSecho to
*not* forward circular messages to links. This would not require dupe
checking at all.
This is the fix I asked Rob to do.
Why would he make it an option and not the default behaviour. I can't think of a reason why you want to forward circular messages (for the second or third or ... time).
So to recap: SBBSecho's dupe checking works fine and indeed duplicate
messages are detected *before* sent to links (and dupe messages are
not forwarded). However, a detected circular path prevents any
duplicate message checking and this is the area for potential change
in behavior.
So there is the explaination. Hopefully within a few weeks the new option can be added to SBBScho so all bases are covered. Right now I've addedthe
NOCHECKPATH to see if that at least helps. Just know IF duplicate MSGID are detected, they are stripped before being sent to links.
Strip MSGID's? That would be a very big bug! ;)
messagesSBBSecho only checks for duplicate messages when importing into a
local message base. If you're a hub for an echo, any messages you
receive from one link (up or downlink) will be propagated to the
others without any dupe checking.
The first sentence is accurate: SBBSecho only checks for duplicate
when importing into a local message base.
The second sentence is not accurate. As far as I can tell from looking at the source history, SBBSecho never propagated detected-duplicate messages to other links.
So I think you miss understood what Rob is saying in his reply to you?
Pass-through areas are a different matter. SBBSecho never checks for duplicate messages in pass-through areas.
Why would he make it an option and not the default behaviour. I can't
think of a reason why you want to forward circular messages (for the
second or third or ... time).
Links change. Just because the local node received the message once does not mean all the links did as well.
SBBSecho has operated this way for over 20 years without any
complaint, so I didn't want to just impose new behavior on sysops that upgrade.
I'm fine making the newly-optional behavior the default for
fresh installs.
| Sysop: | MCMLXXIX |
|---|---|
| Location: | Prospect, CT |
| Users: | 325 |
| Nodes: | 10 (0 / 10) |
| Uptime: | 11:32:14 |
| Calls: | 510 |
| Messages: | 220575 |