Error: Verification of digital signature of inbound AS2 message failed message-digest attribute value does not match calculated value

By as2guy on Wed, 05/20/2020 - 17:40

Hello all,

I have 8 working Mendelson AS2 partners set up but this last one is causing issues. We can send to them without issue, but here are the logs when they send to us.

Inbound transmission is a AS2 message [them-us], raw message size: 2.95 KB.
Inbound AS2 message is encrypted.
The inbound AS2 message data has been decrypted using the key "our-key", the encryption algorithm was "3DES", the key encryption algorithm was "RSA".
Inbound AS2 message is signed.
The sender used the algorithm "SHA1" to sign the inbound AS2 message.
Using certificate "their-key" to verify inbound AS2 message signature.
Verification of digital signature of inbound AS2 message failed message-digest attribute value does not match calculated value

I have imported their key the same way as every other partner, recreated the partner config, closed and reopened Mendelson AS2, rebooted the server, tried to partner with them on a different server also running Mendelson AS2, removed the key and partner then rebooted the server and set it all up again. Everything results in the same error you see at the bottom of the logs. Their cert has the right fingerprint and serial number.

Some help on where I can look to troubleshoot or a fix would be much appreciated. Thanks all, and be safe.

Forum
AS2

Comments

Profile picture for user service

Submitted by service on Fri, 05/22/2020 - 14:21

as2guy,

the digital signature ensures that the data is bytewise the same as it is sent on the sender side. If it is changed somethow, e.g. by a FTP process, linux/windows copy process or whatever that changes any byte in the data the signature cann ot be verified. There is no fix to this on your side, you just have to ensure that the data is not changed after it has been signed by your partner.

Regards

Submitted by as2guy on Fri, 05/22/2020 - 19:20

That you for your very helpful reply.

I learned my partner is using Linux. We are on Windows. Is it safe to say this partner is not compatible with us no matter what, and that is why this is failing with this error? We can send to him, but we get the above error when he sends to us.

I understand there is no fix on my side, but to share some data with my manager, where can I compare the sending partner's byte size vs the one we actually receive?

Once again, thanks for helping us out with this.

Submitted by derekmwright on Mon, 01/17/2022 - 03:36

AS2guy, I’m running into this myself with a custom developed AS2 solution. The problem appears to be on the mendelson side as they dont respect RFC822 and are expecting headers within the enveloped message to be in a specific order. Due to this, they try to reconstruct the message and assume an order of headers and calculate the signature digest over that INSTEAD of using the received message verbatim. This modification is breaking otherwise standard implementations of AS2. Not sure how to get this resolved w/ out enterprise support, however it seems I wouldn't trust mendelson opensource out in the wild. Look for a Drummond certified product to ensure interoperability.

Submitted by derekmwright on Thu, 01/27/2022 - 15:27

Looks like I was right on my first message, after much more debugging and test cases, we found that if an MDN response message has a "LF" char to end a line then we get digest issues with Mendelson. So technically, Mendelson is not respecting the raw body as received, byte for byte and checking the digest. "\n" or LF is getting converted to "\r\n" or CRLF which is causing the signature verification to fail.

Profile picture for user service

Submitted by service on Mon, 01/31/2022 - 10:03

derekmwright,

Inbound data is never translated to a String or read line by line, everything is processed as byte array in the mendelson AS2. Please share your ideas where the problem in our processing is, then we could have a look at it.

Regards

Submitted by hasans on Wed, 08/30/2023 - 21:22

I am having the exact same issue with one of my partners.
Inbound MDN state is [processed/Error: integrity-check-failed].
[2:12:08 p.m.] [Inbound MDN details received from Production: "[ReceiveFile -] Error: Unable to verify content integrity: Message digest was encrypted with unknown algorithm.

[2:12:08 p.m.] XcoreXrssbusconnectX160X5976.ej: Unable to verify content integrity: Message digest was encrypted with unknown algorithm.
[2:12:08 p.m.] at XcoreXrssbusconnectX160X5976.aez.q(Unknown Source)
[2:12:08 p.m.] at rssbus.as2portops.ReceiveFile.a(Unknown Source)
[2:12:08 p.m.] at rssbus.as2portops.ReceiveFile.a(Unknown Source)
[2:12:08 p.m.] at rssbus.as2portops.ReceiveFile.exec(Unknown Source)
[2:12:08 p.m.] at XcoreXrssbusconnectX160X5976.uz.a(Unknown Source)
[2:12:08 p.m.] at XcoreXrssbusconnectX160X5976.uz.a(Unknown Source)
[2:12:08 p.m.] at XcoreXrssbusconnectX160X5976.uz.a(Unknown Source)
[2:12:08 p.m.] at XcoreXrssbusconnectX160X5976.bvo.a(Unknown Source)
[2:12:08 p.m.] at rssbus.pub.Receive_rsb.a(Unknown Source)
[2:12:08 p.m.] at XcoreXrssbusconnectX160X5976.asm.Exec(Unknown Source)
[2:12:08 p.m.] at rssbus.pub.Receive_rsb.execRSB(Unknown Source)
[2:12:08 p.m.] at XcoreXrssbusconnectX160X5976.bav.execRSB(Unknown Source)
[2:12:08 p.m.] at XcoreXrssbusconnectX160X5976.duz.a(Unknown Source)
[2:12:08 p.m.] at XcoreXrssbusconnectX160X5976.cts.a(Unknown Source)
[2:12:08 p.m.] at XcoreXrssbusconnectX160X5976.cts.a(Unknown Source)
[2:12:08 p.m.] at XcoreXrssbusconnectX160X5976.cdb.call(Unknown Source)
[2:12:08 p.m.] at XcoreXrssbusconnectX160X5976.bui.call(Unknown Source)
[2:12:08 p.m.] at rssbus.RSBScript.a(Unknown Source)
[2:12:08 p.m.] at rssbus.RSBScript.service(Unknown Source)
[2:12:08 p.m.] at javax.servlet.http.HttpServlet.service(HttpServlet.java:820)
[2:12:08 p.m.] at weblogic.servlet.internal.StubSecurityHelper$ServletServiceAction.run(StubSecurityHelper.java:227)
[2:12:08 p.m.] at weblogic.servlet.internal.StubSecurityHelper.invokeServlet(StubSecurityHelper.java:125)
[2:12:08 p.m.] at weblogic.servlet.internal.ServletStubImpl.execute(ServletStubImpl.java:301)
[2:12:08 p.m.] at weblogic.servlet.internal.ServletStubImpl.execute(ServletStubImpl.java:184)
[2:12:08 p.m.] at weblogic.servlet.internal.WebAppServletContext$ServletInvocationAction.wrapRun(WebAppServletContext.java:3750)
[2:12:08 p.m.] at weblogic.servlet.internal.WebAppServletContext$ServletInvocationAction.run(WebAppServletContext.java:3714)
[2:12:08 p.m.] at weblogic.security.acl.internal.AuthenticatedSubject.doAs(AuthenticatedSubject.java:321)
[2:12:08 p.m.] at weblogic.security.service.SecurityManager.runAs(SecurityManager.java:120)
[2:12:08 p.m.] at weblogic.servlet.internal.WebAppServletContext.securedExecute(WebAppServletContext.java:2283)
[2:12:08 p.m.] at weblogic.servlet.internal.WebAppServletContext.execute(WebAppServletContext.java:2182)
[2:12:08 p.m.] at weblogic.servlet.internal.ServletRequestImpl.run(ServletRequestImpl.java:1491)
[2:12:08 p.m.] at weblogic.work.ExecuteThread.execute(ExecuteThread.java:256)
[2:12:08 p.m.] at weblogic.work.ExecuteThread.run(ExecuteThread.java:221)"
[2:12:08 p.m.] [70317a92a0c0a106_-71124a90_18a231cd5b6_-3579@ ] Inbound MDN is signed (SHA-256).
[2:12:08 p.m.] [70317a92a0c0a106_-71124a90_18a231cd5b6_-3579@ ] Using certificate "as2-app1-lh.gwlnp. .com" to verify inbound MDN signature.
[2:12:08 p.m.] [70317a92a0c0a106_-71124a90_18a231cd5b6_-3579@ ] Verification of digital signature of inbound MDN failed message-digest attribute value does not match calculated value
[2:12:08 p.m.] [70317a92a0c0a106_-71124a90_18a231cd5b6_-3579@ ] Error verifying the senders digital signature: message-digest attribute value does not match calculated value.