feat: Securejoin v3, encrypt all securejoin messages by Hocuri · Pull Request #7754 · chatmail/core

Hocuri · 2026-01-20T13:36:07Z

Close #7396. Before reviewing, you should read the issue description of #7396.
I recommend to review with hidden whitespace changes.

TODO:

Backwards compatibility:
Everything works seamlessly in my tests. If both devices are updated, then the new protocol is used; otherwise, the old protocol is used. If there is a not-yet-updated second device, it will correctly observe the protocol, and mark the chat partner as verified.

Note that I removed the Auto-Submitted: auto-replied header from securejoin messages. We don't need it ourselves, it's a cleartext header that leaks too much information, and I can't see any reason to have it.

src/mimeparser.rs

Hocuri · 2026-01-26T17:26:47Z

src/mimeparser.rs

                Ok(secret)
            })
            .await?;
+        secrets.extend(token::lookup_all(context, token::Namespace::Auth).await?);


There was the idea to use a new table rather than reusing the AUTH token.

I don't think this would be worth it:

It would increase complexity, because in decode_openpgp() and in "step 5+6" in handle_securejoin_handshake(), we would need to lookup tokens from both the old and new table, in order to keep backwards compatibility, in case someone scans a QR code that was generated before the upgrade, because in that case, the AUTH token would only be in the tokens table and not in the new table.

It would be another table to deprecate eventually. As it stands, we can just one day remove all the invitenumbers from tokens, and ignore the namespc column

All in all, we would need to juggle two tables rather than one.

...but it wouldn't be too much effort to switch to a new table, either.

…rt() and set_content_type_to_encrypted() functions

…ot to work yet)

Previously, AUTH tokens were only synchronized if an INVITE token was generated at the same time. This led to several user-visible bugs. Now, we always send a sync message after generating a new AUTH token. Since the INVITE token may have stayed the same, I added a UNIQUE bound to the `tokens` table, in order to prevent saving the same INVITE token many times.

Hocuri · 2026-02-10T18:31:07Z

benches/decrypting.rs

 //! ```
 //!
-//! or, if you want to only run e.g. the 'Decrypt a symmetrically encrypted message' benchmark:
+//! or, if you want to only run e.g. the 'Decrypt and parse a symmetrically encrypted message' benchmark:


I removed the "Decrypt a symmetrically/public-key encrypted message" benchmarks, because I don't need them anymore, it seems unlikely they will be needed soon, and they made it hard to implement the performance improvement in 61ae743

Hocuri · 2026-02-10T18:33:26Z

src/chat/chat_tests.rs


        let request = charlie.pop_sent_msg().await;
-        assert_eq!(request.recipients, "alice@example.org charlie@example.net");
+        assert_eq!(request.recipients, "alice@example.org");


The first two messages of the protocol aren't sent to self anymore, because in many cases, one's own other devices wouldn't be able to decrypt them, anyways.

Hocuri · 2026-02-10T20:06:46Z

src/mimefactory.rs

+                    .into(),
+            ));
+        }
+


I moved this code up, in order to be able to extract the group_headers_by_confidentiality() function.

Hocuri · 2026-02-10T20:44:17Z

src/message.rs

+/// to prevent downloading the same message in the future.
+///
+/// Returns tombstone database row ID.
+pub(crate) async fn insert_tombstone(context: &Context, rfc724_mid: &str) -> Result<MsgId> {


I moved this function from receive_imf.rs to here, so that it can be used from the securejoin code

Hocuri · 2026-02-10T20:45:22Z

src/securejoin.rs

+        context
+            .sync_qr_code_tokens(Some(chat.grpid.as_str()))
+            .await?;
+        context.scheduler.interrupt_smtp().await;


This actually fixes an existing bug:

We now generate a new AUTH token every time a QR code is shown. Therefore, we always need to sync it.

deltachat-jsonrpc/src/api/types/qr.rs

src/mimeparser.rs

iequidoo · 2026-02-24T13:09:30Z

src/mimeparser.rs

+            // and because Alice anyways couldn't verify his signature at this step,
+            // because she doesn't know his public key yet.
+        } else {
+            remove_header(headers, "secure-join-auth", removed);


Btw, why do we need to remove it at all, what security issues does this fix?

I'm not sure. I can't come up with a concrete attack, but it somewhat feels wrong to allow it in an unencrypted message. @link2xt what do you think?

src/mimeparser.rs

src/securejoin.rs

link2xt

Did not review everything, some questions are in #7396 (comment)

link2xt · 2026-02-25T02:36:19Z

src/sql/migrations.rs

+                namespc INTEGER DEFAULT 0,
+                foreign_key TEXT DEFAULT '',
+                token TEXT DEFAULT '' UNIQUE,
+                timestamp INTEGER DEFAULT 0,


Can some or all columns get NOT NULL constraint? This may break some backwards compatibility if backup is imported into older version, this needs to be tested.

If we have defaults, then backwards compatibility problems are unlikely, right? (except if we explicitly tried to insert NULL/None). But yes, I can test this.

I think we can even remove the foreign_id column, because it wasn't used when we last increased the DCBACKUP_VERSION (it has been unused for over a year now).

OK, I tested that with 805e5ff, everything works fine when downgrading to v2.33.0 (the oldest version since the last increase of DCBACKUP_VERSION):

Securejoin works after downgrading

Starting a securejoin before the downgrade, and finishing it afterwards, works (of course it doesn't work if the other side wants to use v3 because the QR code said v=3, but that's a veeery edge case)

...both for groups and 1:1 chats.

src/sql/migrations.rs

link2xt · 2026-02-25T02:50:27Z

src/chat/chat_tests.rs


    tcm.section("Bob's second device also receives these messages");
-    bob1.recv_msg_trash(&auth_required).await;
+    bob1.recv_msg_trash(&vc_pubkey).await;


Can old devices handle vc-pubkey or will they get undecryptable message if Bob has not upgraded all devices at once?

They will get an undecryptable message, but this message will just be trashed ignored. There will only be a warning in the log, nothing more.

2026-02-25T11:21:33.654Z core/event INFO "" 1 "src/imap/idle.rs:87: \"INBOX\": Idle has NewData ResponseData { raw: 4096, response: MailboxData(Exists(653)) }" 2026-02-25T11:21:33.698Z core/event INFO "" 1 "src/scheduler.rs:425: IMAP loop iteration for inbox finished, keeping the session." 2026-02-25T11:21:33.699Z core/event INFO "" 1 "src/imap.rs:606: fetch_new_msg_batch(INBOX): UIDVALIDITY=1770209638, UIDNEXT=4193." 2026-02-25T11:21:33.756Z core/event INFO "" 1 "src/imap.rs:1386: Starting a full FETCH of message set \"4193\"." 2026-02-25T11:21:33.802Z core/event INFO "" 1 "src/imap.rs:1495: Passing message UID 4193 to receive_imf()." 2026-02-25T11:21:33.803Z core/event WARNING "" 1 "src/mimeparser.rs:404: decryption failed: missing key" 2026-02-25T11:21:33.804Z core/event INFO "" 1 "src/receive_imf.rs:537: Receiving message \"44915ad4-6701-4ffb-a690-dfa397708b35@localhost\", seen=false..." 2026-02-25T11:21:33.804Z core/event INFO "" 1 "src/receive_imf.rs:663: Chat assignment is OneOneChat." 2026-02-25T11:21:33.805Z core/event INFO "" 1 "src/receive_imf.rs:1625: No chat id for message (TRASH)." 2026-02-25T11:21:33.809Z core/event INFO "" 1 "src/receive_imf.rs:2202: Message has 1 parts and is assigned to chat #Chat#Trash." 2026-02-25T11:21:33.810Z core/event INFO "" 1 "src/imap.rs:1540: Successfully received 1 UIDs." 2026-02-25T11:21:33.811Z core/event INFO "" 1 "src/imap.rs:799: 1 mails read from \"INBOX\"." 2026-02-25T11:21:33.949Z core/event INFO "" 1 "src/scheduler.rs:691: IMAP session in folder \"INBOX\" supports IDLE, using it." 2026-02-25T11:21:33.995Z core/event INFO "" 1 "src/imap/idle.rs:64: IDLE entering wait-on-remote state in folder \"INBOX\"."

src/token.rs

link2xt · 2026-02-25T03:05:27Z

src/token.rs

    token: &str,
    timestamp: i64,
 ) -> Result<()> {
+    if token.is_empty() {


Why does it happen? Should it be ensure_and_debug_assert instead?

In the future, we may want to remove the INVITENUMBER from QR codes, because it's not needed anymore. Then, reviving a QR code will try to save an empty INVITENUMBER, so we need some logic for this. test_withdraw_verifycontact_without_invite tests this.

link2xt · 2026-02-25T03:16:22Z

src/securejoin.rs

        })
    } else if let Some(step) = mime_message.get_header(HeaderDef::SecureJoin) {
        match step {
+            "vc-request-pubkey" => Some(SecureJoinStep::RequestPubkey),


Can we name the new messages request-pubkey and pubkey? There is no vg variant, even for legacy messages it is not used anymore.

Not sure, then we have inconsistency between request-pubkey/pubkey and the others. Also, I really liked that I immediately knew that if something starts with vc-, it's a securejoin step.

OTOH, it's not important to me; if you insist, I'll just rename them.

src/receive_imf.rs

src/securejoin.rs

Co-authored-by: iequidoo <117991069+iequidoo@users.noreply.github.com>

…date

benches/decrypting.rs

deltachat-jsonrpc/src/api/types/qr.rs

iequidoo · 2026-02-25T12:28:46Z

src/mimefactory.rs

+        mail_builder::headers::address::Address::new_list(to.clone()).into(),
+    ));
+
+    headers.push((


Why do we need it? Subject is an optional header

Not sure, maybe some MTAs reject messages without a subject. @link2xt ?

group_headers_by_confidentiality() will only insert Subject: [...] in the unencrypted part if there is also an encrypted subject, but of course we could just add Subject: [...] to the unencrypted headers after the group_headers_by_confidentiality() call

All our messages have Subject: [...]. I don't know which MTAs reject such messages, maybe it theoretically can be removed, but practically it is likely to trigger some filters.

but of course we could just add Subject: [...] to the unencrypted headers after the group_headers_by_confidentiality() call

Yes, i only meant that encrypted Subject can be omitted. But if that is complicated code-wise, it can be left for now

src/securejoin/securejoin_tests.rs

src/mimefactory.rs

src/sql/migrations.rs

Hocuri changed the title ~~[WIP] Securejoin v3: Encrypt all securejoin messages~~ [WIP] feat: Securejoin v3, encrypt all securejoin messages Jan 20, 2026

Hocuri force-pushed the hoc/securejoin-v3 branch 2 times, most recently from 59957ed to ef97a7e Compare January 21, 2026 15:44

Hocuri commented Jan 22, 2026

View reviewed changes

src/mimeparser.rs Outdated Show resolved Hide resolved

Hocuri force-pushed the hoc/securejoin-v3 branch from 807b8c4 to afb7e2a Compare January 22, 2026 17:06

Hocuri commented Jan 26, 2026

View reviewed changes

Hocuri force-pushed the hoc/securejoin-v3 branch from afb7e2a to 81f79a9 Compare January 26, 2026 18:18

Hocuri added 19 commits January 27, 2026 12:31

Send v=3 in QR codes

c0b86f5

[wip, not compiling] Add render_symm_encrypted_securejoin_message()

ad584d2

[wip, not compiling] extract function group_headers_by_confidentiality()

3b66611

[wip, doesn't compile] refactor: exctract add_headers_to_encrypted_pa…

2025e9b

…rt() and set_content_type_to_encrypted() functions

[wip, doesn't compile] fix: Set subject for securejoin message

2eba552

[wip, doesn't compile] Extract render_outer_message() functions

2566c5b

It compiles

1014bf1

feat: Decrypt and answer on Alice's side (sending the message seems n…

c91b349

…ot to work yet)

Fix some bugs when sending vc-pubkey, decrypt vc-pubkey on Bob's side

4c00bc1

Handle vc-pubkey on Bob's side

d979938

Adapt some tests

c9bd5d0

fix: Don't leak cryptographic identity by signing vc-request-pubkey

c3d4c43

fix: Don't limit number of auth tokens to 1

7ff5cd6

modify test

297c6d8

Update golden tests

2358c9d

Make the remaining Rust tests pass

90b0f94

Update src/mimeparser.rs

aceb75b

Add failing test test_auth_token_is_synchronized()

8ee3c2a

Hocuri force-pushed the hoc/securejoin-v3 branch from 81f79a9 to 6b658a0 Compare January 30, 2026 16:46

Hocuri added 3 commits February 10, 2026 10:52

Merge remote-tracking branch 'origin/main' into hoc/securejoin-v3

db7e667

bench: Also add auth tokens into benchmark

20ef54b

bench: Add black_box

50eb923

Hocuri added 2 commits February 10, 2026 20:13

test: Improve test

10933cc

Some more comments improvements

fd61423

Hocuri force-pushed the hoc/securejoin-v3 branch from 55d6919 to fd61423 Compare February 10, 2026 19:21

Hocuri commented Feb 10, 2026

View reviewed changes

More comments and renamings

0d47a52

Hocuri changed the title ~~[WIP] feat: Securejoin v3, encrypt all securejoin messages~~ feat: Securejoin v3, encrypt all securejoin messages Feb 10, 2026

Hocuri requested review from iequidoo and link2xt February 10, 2026 20:52

Merge remote-tracking branch 'origin/main' into hoc/securejoin-v3

3b0f798

Hocuri mentioned this pull request Feb 23, 2026

fix: Handle the case that the user starts a securejoin, and then deletes the contact #7883

Open

Merge remote-tracking branch 'origin/main' into hoc/securejoin-v3

fe3c477

iequidoo reviewed Feb 24, 2026

View reviewed changes

link2xt reviewed Feb 25, 2026

View reviewed changes

Hocuri and others added 5 commits February 25, 2026 09:00

Update deltachat-jsonrpc/src/api/types/qr.rs

dc414a6

Co-authored-by: iequidoo <117991069+iequidoo@users.noreply.github.com>

iequidoo's review

b29a45b

Import insert_tombstone()

57ec3ac

link2xt's review

6be7491

We do need to handle empty tokens in token::save()

3e6083a

Hocuri force-pushed the hoc/securejoin-v3 branch from 730da02 to 3e6083a Compare February 25, 2026 09:43

Hocuri added 2 commits February 25, 2026 11:22

Fix: Don't fail when there is an existing bobstate from before the up…

31a6b90

…date

refactor: Don't load id in load_shared_secrets(), we don't need it

4295c7a

Hocuri force-pushed the hoc/securejoin-v3 branch from 5b2faef to 4295c7a Compare February 25, 2026 10:22

refactor: Remove foreign_id column from tokens, make everything NOT NULL

805e5ff

iequidoo reviewed Feb 25, 2026

View reviewed changes

"sender" -> "inviter"

3ecc077

iequidoo approved these changes Feb 26, 2026

View reviewed changes

link2xt approved these changes Feb 28, 2026

View reviewed changes

src/sql/migrations.rs Outdated Show resolved Hide resolved

src/sql/migrations.rs Outdated Show resolved Hide resolved

Hocuri added 3 commits February 28, 2026 21:51

Merge remote-tracking branch 'origin/main' into hoc/securejoin-v3

2678dd8

Adapt function documentation

98c3ca9

Remove default for namespace

0c2821b

Uh oh!

Conversation

Hocuri commented Jan 20, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Uh oh!

Hocuri Jan 26, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

link2xt left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Hocuri Feb 25, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Hocuri Feb 25, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Hocuri commented Jan 20, 2026 •

edited

Loading

Hocuri Jan 26, 2026 •

edited

Loading

Hocuri Feb 25, 2026 •

edited

Loading

Hocuri Feb 25, 2026 •

edited

Loading