Creating new tokens does not work with upgrade to 5.2.1 (original) (raw)

my workflow before the update was that I simply called /hub/api/users/${username}/tokens with the username and password in the body of the POST which would give me a token that I could use. Now when I try calling it, I get a "'_xsrf' argument missing from POST" 403 error. Is there another method I should be using to create a token and or login?

minrk April 23, 2025, 7:35am 2

Sometimes auth errors get falsely reported as xsrf errors. Can you share more about how you were making the request?

This request works with jupyterhub 5.3.0 (using dummy authenticator, so any password accepted):

curl -d '{"auth": {"username": "test", "password": "dummy"}}' -X POST http://127.0.0.1:8765/hub/api/users/test/tokens

gives

{
  "user": "test",
  "id": "a65",
  "kind": "api_token",
  "roles": [],
  "scopes": [
    "read:shares!user=test",
    "users:activity!user=test",
    "read:tokens!user=test",
    "read:users:name!user=test",
    "servers!user=test",
    "read:users!user=test",
    "read:users:shares!user=test",
    "delete:servers!user=test",
    "users:shares!user=test",
    "access:services",
    "read:users:activity!user=test",
    "access:servers!user=test",
    "read:users:groups!user=test",
    "tokens!user=test",
    "read:servers!user=test"
  ],
  "created": "2025-04-23T07:32:34.999533Z",
  "last_activity": null,
  "expires_at": null,
  "note": "Requested via api",
  "session_id": null,
  "oauth_client": "JupyterHub",
  "token": "74fbbe18542046b6a898e842145c6672"
}

minrk April 23, 2025, 7:35am 3

Oh, also the Hub logs preceeding the failed request may point to the real error

scalper April 23, 2025, 3:01pm 4

what I’ve noticed is that basically, if I’m logged into the hub and I try to call the tokens endpoint from a different domain I get the xsrf error. if I sign out then retry it works. so I think the error is valid at this point. How hard is it to use an actual authenticator and not the dummy?

also, I’ve noticed that the login and logout apis aren’t part of the docs JupyterHub REST API — JupyterHub documentation
is there a reason why? my workflow is basically I try to create a token → fails if user is logged in → logout → then create the token again .

also, I see mixed wording on how to start a server. the docs state to use /hub/api/users/{name}/server but I’ve also seen this error Error: Could not get list of available scripts: JupyterHub server no longer running at /user/noauth/api/contents/examples/notebook_examples. Restart the server at http://xxx.xxx/hub/spawn/noauth

which endpoint should I use to start a server?

minrk April 23, 2025, 5:39pm 5

Yes, it works with PAM, too, I just can’t share the password. The Authenticator is not related to the xsrf check.

Ah, I didn’t realize this was an actual cross-site request instead of from the backend without cookies. Then I need to look into whether the xsrf check gets applied to the attempted cookie auth before checking the auth body. The cookie shouldn’t be accepted, but that shouldn’t prevent the body from working, which is what sounds like might be happening.

Yes, because login and logout are not part of the API. You can’t logout with the API. These are browser endpoints (html pages for humans), not api endpoints.

/hub/api/users/{name}/server is the API endpoint for starting a server. /hub/spawn/ is the browser endpoint (html page with form, redirect, etc.). I’ll think about how this error might be clearer in the two contexts.

scalper April 23, 2025, 5:57pm 6

my workaround for this xsrf issue is to call GET /hub/login/ and then the response has the xsrf embedded inside. then recall the tokens POST. this is hacky at best… but it’s what my app needs

also, I can logout using the /hub/logout endpoint. maybe it’s just the hub logging out and not the user? is that possible?

minrk April 23, 2025, 6:04pm 7

Can you share some of your jupyterhub config, especially anything related to xsrf? And hub logs surrounding one of these sequences?

Does your app have any backend component, or purely in the browser?

scalper April 23, 2025, 6:07pm 8

I unfortunately do not have the logs, I could prob ask around for the jupyterhub config but it might take awhile. Yes we do have a backend component.

While you’re here, I was curious on why creating a token does not require a username and password? before updating the server, I believe JupyterHub 1.2.1 or something required a username and password, now it only needs username. so technically it sounds like you can create a token and run jupyterhub APIs without ever needing a password?

minrk April 23, 2025, 6:18pm 9

What is the URL of your service relative and the hub (hostname, path, etc.)?

scalper April 23, 2025, 6:28pm 10

so I have a proxy that calls the server that the jupyterhub is on. but since it originates from the browser on a different host, it has issues.

scalper April 23, 2025, 6:31pm 11

another question, when I execute a proc, and I’m not logged in but pass a token, it still requires me to login. whereas my old server didn’t, is this intended?

minrk April 23, 2025, 7:18pm 12

I’m struggling to understand the flow of requests and credentials, can you share more about the sequence of requests and what credentials are where?

It is quite unusual to be making xs requests to the hub without oauth tokens, which eliminate all xsrf questions.

It would all work much smoother, I think, if the backend were an appropriately permissioned Service and made direct api requests with an api token, rather than relying on cross-site browser requests, which mostly shouldn’t work.

Yes, this is disabled by default because forced login is insecure. You can enable it by setting c.HubAuth.allow_token_in_url = True in your single-user environment (e.g. /etc/jupyter/jupyter_server_config.py in a user image or container).

scalper April 23, 2025, 10:12pm 13

sorry. I’ll try my best to explain our workflow.

we have a web app that’s proxied through a gateway that we maintain. this gateway makes our API calls for us. in this case, the web app calls an endpoint which calls the jupyterhub server via this gateway.

my workflow in the web app goes like this - I create a token, this fails if a user is already logged in. if this happens, we simply logout the user using hub/logout and recreate the token.

now when I go to execute a proc, I have to login. logging in before creating the token doesn’t work because the xsrf don’t match.

so if login are not part of the API, how does a user log into the jupyterhub server? using jupyterhub GUI

minrk April 24, 2025, 8:51am 14

What credentials does the gateway use to make requests to the Hub, and what Authenticator are you using? How do the users login to the Hub? If you’re logging in, it sounds like the proxy is using browser cookies to try to authenticate requests to the Hub, which is going to get complex. I’d recommend issuing a token for the Gateway itself and making token-authenticated requests instead of relying on cookies.

It seems like perhaps you should be using the API more, possibly bypassing JupyterHub UI altogether using api only mode, and going straight to the user servers.

scalper April 24, 2025, 3:49pm 15

okay, I’m trying to wrap my head around that. I’d recommend issuing a token for the Gateway itself and making token-authenticated requests instead of relying on cookies.

how would this look like? would I still have to create a token? or would any API call I make have a token created by the gateway which would then pass it to jupyterhub?

minrk April 25, 2025, 6:34am 16

Generate a token out of band (e.g. openssl rand -hex 32) and register your gateway as a service with specific permissions:

# register the token and service
c.JupyterHub.services = [
    {
        "name": "my-gateway",
        "api_token": "...",
    }
]

# grant the service permissions to do what it should do
c.JupyterHub.load_roles = [
    {
        "name": "gateway",
        "scopes": [
            "read:users", # read info about users
            "servers", # start/stop servers
            "tokens", # if the gateway still needs to issue tokens for users, that's unclear to me
        ],
        "services": ["my-gateway"],
    }
]

Then all API calls from the gateway are made with the header Authorization: Bearer ${gateway-token}.