fix(codeintel): improve upload error hints for auth and permission failures

[chrapkowski-sg]

When upload of a scip index fails, the error message may be misleading or incomplete. The complexity comes from two aspects. First that we have 2 tokens - sourcegraph access token and code host token. This PR makes sure that whenever a token is invalid we get most probable causes. Second part is that even if a token is valid on its own, the user may not have sufficient permission. https://github.com/sourcegraph/sourcegraph/pull/10077 makes the error codes and messages more consistent on the server side. This way user gets a hint if token was wrong or there was a permission problem.

To summarise. First of all user gets error message from the server which often is specific (ex. upload failed: must provide gitlab_token). However basing text returned by the server (server does not return structured response like JSON) is IMO fragile so except this error message user gets hints about possible problems.

I believe those two together should eliminate confusion and point user in the right direction how to remediate the issue. We also always show the link to the documentation.

Additionally existing code was deducing used code host based on the repository URL hostname, here we also check the specified command line parameter. Those hints which are displayed to the user is not a new invention, those were just improved to provide user with more accurate information and to utilise information from the server which effectively is best source of truth what the problem was.

Test plan

Unit tests and additional integration tests were performed to make sure the provided hints work as expected

==============================================================================
[PASS ✅] 1. GitLab: no gitlab-token flag
  Valid SG token, gitlab.com repo, no -gitlab-token flag.
  Server rejects with 401 'must provide gitlab_token'.
  Hints: SG token valid-format (is401) + no gitlab-token for gitlab.com repo.

  Command:
    SRC_ACCESS_TOKEN=<valid SG token> \
          src code-intel upload \
          -repo gitlab.com/go/pentagi \
          -commit 5467af29c153... \
          -insecure-skip-verify

  Exit code: 1 (expected non-zero)

  Output:
    💡 Inferred arguments
       repo: gitlab.com/go/pentagi
       commit: 5467af29c153c96f7808ee2f318c839e1b155f12
       root:
       file: index.scip
       indexer: scip-go
       indexerVersion: 0.1.26
    ✅ Index compressed
    💡 Indexed compressed (0.00MB -> 0.00MB).
    ❌ Failed to upload index file
    upload failed: must provide gitlab_token

    Possible causes:
    - The Sourcegraph access token may be invalid, expired, or you may be connecting to the wrong Sourcegraph instance.
    - No -gitlab-token was provided. If this Sourcegraph instance enforces code host authentication, retry with -gitlab-token=<token> for a token with access to gitlab.com/go/pentagi.

    For more details on uploading SCIP indexes, see https://sourcegraph.com/docs/cli/references/code-intel/upload.
==============================================================================

==============================================================================
[PASS ✅] 2. GitLab: empty gitlab-token
  Valid SG token, gitlab.com repo, -gitlab-token=''.
  Server treats empty token same as missing → 401 'must provide gitlab_token'.
  Hints: same as #1 (empty string not counted as 'supplied').

  Command:
    SRC_ACCESS_TOKEN=<valid SG token> \
          src code-intel upload \
          -repo gitlab.com/go/pentagi \
          -commit 5467af29c153... \
          -gitlab-token "" \
          -insecure-skip-verify

  Exit code: 1 (expected non-zero)

  Output:
    💡 Inferred arguments
       repo: gitlab.com/go/pentagi
       commit: 5467af29c153c96f7808ee2f318c839e1b155f12
       root:
       file: index.scip
       indexer: scip-go
       indexerVersion: 0.1.26
    ✅ Index compressed
    💡 Indexed compressed (0.00MB -> 0.00MB).
    ❌ Failed to upload index file
    upload failed: must provide gitlab_token

    Possible causes:
    - The Sourcegraph access token may be invalid, expired, or you may be connecting to the wrong Sourcegraph instance.
    - No -gitlab-token was provided. If this Sourcegraph instance enforces code host authentication, retry with -gitlab-token=<token> for a token with access to gitlab.com/go/pentagi.

    For more details on uploading SCIP indexes, see https://sourcegraph.com/docs/cli/references/code-intel/upload.
==============================================================================

==============================================================================
[PASS ✅] 3. GitLab: invalid gitlab-token
  Valid SG token, gitlab.com repo, -gitlab-token=<invalid>.
  Server forwards token to GitLab API → GitLab rejects → 401.
  Hints: SG token valid-format (is401) + gitlab-token supplied.

  Command:
    SRC_ACCESS_TOKEN=<valid SG token> \
          src code-intel upload \
          -repo gitlab.com/go/pentagi \
          -commit 5467af29c153... \
          -gitlab-token <invalid GitLab token> \
          -insecure-skip-verify

  Exit code: 1 (expected non-zero)

  Output:
    💡 Inferred arguments
       repo: gitlab.com/go/pentagi
       commit: 5467af29c153c96f7808ee2f318c839e1b155f12
       root:
       file: index.scip
       indexer: scip-go
       indexerVersion: 0.1.26
    ✅ Index compressed
    💡 Indexed compressed (0.00MB -> 0.00MB).
    Upload: 99%
    Upload: 99%
    Upload: 99%
    Upload: 99%
    Upload: 99%
    ❌ Failed to upload index file
    upload failed: gitlab error: http status 401: {"message":"401 Unauthorized"}

    Possible causes:
    - The Sourcegraph access token may be invalid, expired, or you may be connecting to the wrong Sourcegraph instance.
    - The supplied -gitlab-token may be invalid.

    For more details on uploading SCIP indexes, see https://sourcegraph.com/docs/cli/references/code-intel/upload.
==============================================================================

==============================================================================
[PASS ✅] 4. GitLab: valid token, no project access
  Valid SG token, gitlab.com repo, -gitlab-token=<valid, no access>.
  Server verifies token with GitLab → user lacks write permission → 403.
  Hints: is403 → 'may not have sufficient permissions' + gitlab token lacks permissions.

  Command:
    SRC_ACCESS_TOKEN=<valid SG token> \
          src code-intel upload \
          -repo gitlab.com/go/pentagi \
          -commit 5467af29c153... \
          -gitlab-token <GitLab token without project access> \
          -insecure-skip-verify

  Exit code: 1 (expected non-zero)

  Output:
    💡 Inferred arguments
       repo: gitlab.com/go/pentagi
       commit: 5467af29c153c96f7808ee2f318c839e1b155f12
       root:
       file: index.scip
       indexer: scip-go
       indexerVersion: 0.1.26
    ✅ Index compressed
    💡 Indexed compressed (0.00MB -> 0.00MB).
    ❌ Failed to upload index file
    upload failed: you do not have write permission to this GitLab project

    Possible causes:
    - You may not have sufficient permissions on this Sourcegraph instance.
    - The supplied -gitlab-token may lack the required permissions.

    For more details on uploading SCIP indexes, see https://sourcegraph.com/docs/cli/references/code-intel/upload.
==============================================================================

==============================================================================
[PASS ✅] 5. GitLab: valid token with project access
  Valid SG token, gitlab.com repo, -gitlab-token=<valid, with access>.
  Server verifies token → user has write permission → upload succeeds.

  Command:
    SRC_ACCESS_TOKEN=<valid SG token> \
          src code-intel upload \
          -repo gitlab.com/go/pentagi \
          -commit 5467af29c153... \
          -gitlab-token <GitLab token with project access> \
          -insecure-skip-verify

  Exit code: 0 (expected 0)

  Output:
    💡 Inferred arguments
       repo: gitlab.com/go/pentagi
       commit: 5467af29c153c96f7808ee2f318c839e1b155f12
       root:
       file: index.scip
       indexer: scip-go
       indexerVersion: 0.1.26
    ✅ Index compressed
    💡 Indexed compressed (0.00MB -> 0.00MB).
    ✅ Index uploaded
    💡 View processing status at https://sourcegraph.test:3443/gitlab.com/go/pentagi/-/code-intelligence/uploads/U0NJUFVwbG9hZDo0MTg4
==============================================================================

==============================================================================
[PASS ✅] 6. GitHub: no github-token flag
  Valid SG token, github.com repo, no -github-token flag.
  Server rejects with 401 'must provide github_token'.
  Hints: SG token valid-format (is401) + no github-token for github.com repo.

  Command:
    SRC_ACCESS_TOKEN=<valid SG token> \
          src code-intel upload \
          -repo github.com/chrapkowski-sg/fmt \
          -commit 605dd2a1d99c... \
          -insecure-skip-verify

  Exit code: 1 (expected non-zero)

  Output:
    💡 Inferred arguments
       repo: github.com/chrapkowski-sg/fmt
       commit: 605dd2a1d99c78e2d7f9662beab60c4a25af6941
       root:
       file: index.scip
       indexer: scip-clang
       indexerVersion: 0.3.3
    ✅ Index compressed
    💡 Indexed compressed (12.29MB -> 1.91MB).
    ❌ Failed to upload index file
    upload failed: must provide github_token

    Possible causes:
    - The Sourcegraph access token may be invalid, expired, or you may be connecting to the wrong Sourcegraph instance.
    - No -github-token was provided. If this Sourcegraph instance enforces code host authentication, retry with -github-token=<token> for a token with access to github.com/chrapkowski-sg/fmt.

    For more details on uploading SCIP indexes, see https://sourcegraph.com/docs/cli/references/code-intel/upload.
==============================================================================

==============================================================================
[PASS ✅] 7. GitHub: empty github-token
  Valid SG token, github.com repo, -github-token=''.
  Server treats empty token same as missing → 401 'must provide github_token'.
  Hints: same as #6 (empty string not counted as 'supplied').

  Command:
    SRC_ACCESS_TOKEN=<valid SG token> \
          src code-intel upload \
          -repo github.com/chrapkowski-sg/fmt \
          -commit 605dd2a1d99c... \
          -github-token "" \
          -insecure-skip-verify

  Exit code: 1 (expected non-zero)

  Output:
    💡 Inferred arguments
       repo: github.com/chrapkowski-sg/fmt
       commit: 605dd2a1d99c78e2d7f9662beab60c4a25af6941
       root:
       file: index.scip
       indexer: scip-clang
       indexerVersion: 0.3.3
    ✅ Index compressed
    💡 Indexed compressed (12.29MB -> 1.91MB).
    ❌ Failed to upload index file
    upload failed: must provide github_token

    Possible causes:
    - The Sourcegraph access token may be invalid, expired, or you may be connecting to the wrong Sourcegraph instance.
    - No -github-token was provided. If this Sourcegraph instance enforces code host authentication, retry with -github-token=<token> for a token with access to github.com/chrapkowski-sg/fmt.

    For more details on uploading SCIP indexes, see https://sourcegraph.com/docs/cli/references/code-intel/upload.
==============================================================================

==============================================================================
[PASS ✅] 8. GitHub: invalid github-token
  Valid SG token, github.com repo, -github-token=<invalid>.
  Server forwards token to GitHub API → GitHub rejects → 401.
  Hints: SG token valid-format (is401) + github-token supplied.

  Command:
    SRC_ACCESS_TOKEN=<valid SG token> \
          src code-intel upload \
          -repo github.com/chrapkowski-sg/fmt \
          -commit 605dd2a1d99c... \
          -github-token <invalid GitHub token> \
          -insecure-skip-verify

  Exit code: 1 (expected non-zero)

  Output:
    💡 Inferred arguments
       repo: github.com/chrapkowski-sg/fmt
       commit: 605dd2a1d99c78e2d7f9662beab60c4a25af6941
       root:
       file: index.scip
       indexer: scip-clang
       indexerVersion: 0.3.3
    ✅ Index compressed
    💡 Indexed compressed (12.29MB -> 1.91MB).
    ❌ Failed to upload index file
    upload failed: github error: Bad credentials

    Possible causes:
    - The Sourcegraph access token may be invalid, expired, or you may be connecting to the wrong Sourcegraph instance.
    - The supplied -github-token may be invalid.

    For more details on uploading SCIP indexes, see https://sourcegraph.com/docs/cli/references/code-intel/upload.
==============================================================================

==============================================================================
[PASS ✅] 9. GitHub: valid token, no repo access
  Valid SG token, github.com repo, -github-token=<valid, no access>.
  Server verifies token with GitHub → user lacks access → 403.
  Hints: is403 → 'may not have sufficient permissions' + github docs.

  Command:
    SRC_ACCESS_TOKEN=<valid SG token> \
          src code-intel upload \
          -repo github.com/chrapkowski-sg/fmt \
          -commit 605dd2a1d99c... \
          -github-token <GitHub token without repo access> \
          -insecure-skip-verify

  Exit code: 1 (expected non-zero)

  Output:
    💡 Inferred arguments
       repo: github.com/chrapkowski-sg/fmt
       commit: 605dd2a1d99c78e2d7f9662beab60c4a25af6941
       root:
       file: index.scip
       indexer: scip-clang
       indexerVersion: 0.3.3
    ✅ Index compressed
    💡 Indexed compressed (12.29MB -> 1.91MB).
    ❌ Failed to upload index file
    upload failed: you do not have write permission to this GitHub repository

    Possible causes:
    - You may not have sufficient permissions on this Sourcegraph instance.
    - The supplied -github-token may lack the required permissions.

    For more details on uploading SCIP indexes, see https://sourcegraph.com/docs/cli/references/code-intel/upload.
==============================================================================

==============================================================================
[PASS ✅] 10. GitHub: valid token with repo access
  Valid SG token, github.com repo, -github-token=<valid, with access>.
  Server verifies token → user has access → upload succeeds.

  Command:
    SRC_ACCESS_TOKEN=<valid SG token> \
          src code-intel upload \
          -repo github.com/chrapkowski-sg/fmt \
          -commit 605dd2a1d99c... \
          -github-token <GitHub token with repo access> \
          -insecure-skip-verify

  Exit code: 0 (expected 0)

  Output:
    💡 Inferred arguments
       repo: github.com/chrapkowski-sg/fmt
       commit: 605dd2a1d99c78e2d7f9662beab60c4a25af6941
       root:
       file: index.scip
       indexer: scip-clang
       indexerVersion: 0.3.3
    ✅ Index compressed
    💡 Indexed compressed (12.29MB -> 1.91MB).
    ✅ Index uploaded
    💡 View processing status at https://sourcegraph.test:3443/github.com/chrapkowski-sg/fmt/-/code-intelligence/uploads/U0NJUFVwbG9hZDo0MTg5
==============================================================================

==============================================================================
[PASS ✅] 11. No SRC_ACCESS_TOKEN, gitlab.com repo
  SRC_ACCESS_TOKEN not set at all, gitlab.com repo, no code-host token.
  Server rejects at SG auth layer → 401.
  Hints: is401 + empty token → 'No Sourcegraph access token was provided.'

  Command:
    # SRC_ACCESS_TOKEN not set \
          src code-intel upload \
          -repo gitlab.com/go/pentagi \
          -commit 5467af29c153... \
          -insecure-skip-verify

  Exit code: 1 (expected non-zero)

  Output:
    💡 Inferred arguments
       repo: gitlab.com/go/pentagi
       commit: 5467af29c153c96f7808ee2f318c839e1b155f12
       root:
       file: index.scip
       indexer: scip-go
       indexerVersion: 0.1.26
    ✅ Index compressed
    💡 Indexed compressed (0.00MB -> 0.00MB).
    ❌ Failed to upload index file
    upload failed: no current user

    Possible causes:
    - No Sourcegraph access token was provided. Set the SRC_ACCESS_TOKEN environment variable to a valid token.
    - No -gitlab-token was provided. If this Sourcegraph instance enforces code host authentication, retry with -gitlab-token=<token> for a token with access to gitlab.com/go/pentagi.

    For more details on uploading SCIP indexes, see https://sourcegraph.com/docs/cli/references/code-intel/upload.
==============================================================================

==============================================================================
[PASS ✅] 12. Invalid format SRC_ACCESS_TOKEN, github.com repo
  SRC_ACCESS_TOKEN='this-is-not-a-valid-sgp-token', github.com repo, no code-host token.
  Server rejects at SG auth layer → 401.
  Hints: is401 + bad format → 'does not match the expected format ... Was it copied incorrectly or truncated?'
  Paragraph 2: no github-token for github.com repo → 'No -github-token was provided'.

  Command:
    SRC_ACCESS_TOKEN=<invalid format SG token> \
          src code-intel upload \
          -repo github.com/chrapkowski-sg/fmt \
          -commit 605dd2a1d99c... \
          -insecure-skip-verify

  Exit code: 1 (expected non-zero)

  Output:
    💡 Inferred arguments
       repo: github.com/chrapkowski-sg/fmt
       commit: 605dd2a1d99c78e2d7f9662beab60c4a25af6941
       root:
       file: index.scip
       indexer: scip-clang
       indexerVersion: 0.3.3
    ✅ Index compressed
    💡 Indexed compressed (12.29MB -> 1.91MB).
    ❌ Failed to upload index file
    upload failed: Invalid access token.

    Possible causes:
    - The provided Sourcegraph access token does not match the expected format (sgp_<40 hex chars> or sgp_<instance-id>_<40 hex chars>). Was it copied incorrectly or truncated?
    - No -github-token was provided. If this Sourcegraph instance enforces code host authentication, retry with -github-token=<token> for a token with access to github.com/chrapkowski-sg/fmt.

    For more details on uploading SCIP indexes, see https://sourcegraph.com/docs/cli/references/code-intel/upload.
==============================================================================

==============================================================================
[PASS ✅] 13. Revoked SG token + github-token, github.com repo
  SRC_ACCESS_TOKEN=<valid format but revoked>, github.com repo, -github-token=<invalid>.
  Server rejects at SG auth layer (token not found) → 401.
  Hints: is401 + valid format → 'Sourcegraph access token may be invalid, expired'
  Paragraph 2: github-token supplied → 'supplied -github-token may be invalid'.

  Command:
    SRC_ACCESS_TOKEN=<revoked SG token (valid format)> \
          src code-intel upload \
          -repo github.com/chrapkowski-sg/fmt \
          -commit 605dd2a1d99c... \
          -github-token <invalid GitHub token> \
          -insecure-skip-verify

  Exit code: 1 (expected non-zero)

  Output:
    💡 Inferred arguments
       repo: github.com/chrapkowski-sg/fmt
       commit: 605dd2a1d99c78e2d7f9662beab60c4a25af6941
       root:
       file: index.scip
       indexer: scip-clang
       indexerVersion: 0.3.3
    ✅ Index compressed
    💡 Indexed compressed (12.29MB -> 1.91MB).
    ❌ Failed to upload index file
    upload failed: Invalid access token.

    Possible causes:
    - The Sourcegraph access token may be invalid, expired, or you may be connecting to the wrong Sourcegraph instance.
    - The supplied -github-token may be invalid.

    For more details on uploading SCIP indexes, see https://sourcegraph.com/docs/cli/references/code-intel/upload.
==============================================================================

==============================================================================
[PASS ✅] 14. Revoked SG token, bitbucket.org repo (catch-all hint)
  SRC_ACCESS_TOKEN=<valid format but revoked>, bitbucket.org repo, no code-host tokens.
  Server rejects at SG auth layer → 401.
  Hints: is401 + valid format → 'Sourcegraph access token may be invalid, expired'.
  Paragraph 2: no tokens + no github.com/gitlab.com prefix → catch-all hint.

  Command:
    SRC_ACCESS_TOKEN=<revoked SG token (valid format)> \
          src code-intel upload \
          -repo bitbucket.org/test/repo \
          -commit 5467af29c153... \
          -insecure-skip-verify

  Exit code: 1 (expected non-zero)

  Output:
    💡 Inferred arguments
       repo: bitbucket.org/test/repo
       commit: 5467af29c153c96f7808ee2f318c839e1b155f12
       root:
       file: index.scip
       indexer: scip-go
       indexerVersion: 0.1.26
    ✅ Index compressed
    💡 Indexed compressed (0.00MB -> 0.00MB).
    ❌ Failed to upload index file
    upload failed: Invalid access token.

    Possible causes:
    - The Sourcegraph access token may be invalid, expired, or you may be connecting to the wrong Sourcegraph instance.
    - Code host verification is supported for github.com and gitlab.com repositories.

    For more details on uploading SCIP indexes, see https://sourcegraph.com/docs/cli/references/code-intel/upload.
==============================================================================

==============================================================================

[chrapkowski-sg]

This change is part of the following stack:

• fix(codeintel): improve upload error hints for auth and permission failures #1259 ◀

_{Change managed by git-spice.}

[keegancsmith]

LGTM

minor: please write a useful PR title before landing. Also when you do land this, your commit message doesn't need to include the large amount of test plan details.

[keegancsmith]

just checking my understanding: so the reason you can't just use errors.As is it will return the first found httpErr but you want to check all of them to find one which could be a 401/403.

[chrapkowski-sg]

Correct. To give more context.

When retries happen, errors get combined into a MultiError. If e.g. the first attempt gets a (502 which did happen to me sometimes during testing) and a retry gets a 401, errors.As returns the first ErrUnexpectedStatusCode it finds (the 502), so the httpErr.Code == 401 || httpErr.Code == 403 check fails — even though an auth error exists in the chain.

[chrapkowski-sg]

LGTM

minor: please write a useful PR title before landing. Also when you do land this, your commit message doesn't need to include the large amount of test plan details.

Good points, thank you!

[chrapkowski-sg]

Removed here handleUploadError at this point only flags validation errors may occur and handleUploadError is about handling errors of during actual upload

[chrapkowski-sg]

Correct. To give more context.

When retries happen, errors get combined into a MultiError. If e.g. the first attempt gets a (502 which did happen to me sometimes during testing) and a retry gets a 401, errors.As returns the first ErrUnexpectedStatusCode it finds (the 502), so the httpErr.Code == 401 || httpErr.Code == 403 check fails — even though an auth error exists in the chain.