feature: Fetch JWT certificates dynamically from a JWKS endpoint, fix: database-agnostic SQL helpers for timestamp formatting#161
Conversation
…ndpoint, with an updated embedded certificate as a fallback.
There was a problem hiding this comment.
Pull request overview
This PR improves cross-database compatibility for time-based metrics queries by introducing driver-aware timestamp SQL expression helpers, and enhances Casdoor JWT verification by attempting to load the public certificate dynamically from a JWKS endpoint at startup (with fallback to the embedded cert).
Changes:
- Add DB-driver-specific SQL helpers for Unix timestamp conversion and time-bucket formatting, and apply them in metrics queries.
- Update time granularity formatting logic to be driver-aware (MySQL/Postgres/MSSQL/SQLite).
- Fetch Casdoor JWT certificate from a JWKS endpoint during
InitCasdoorConfig(), falling back to the embedded certificate on failure.
Reviewed changes
Copilot reviewed 2 out of 2 changed files in this pull request and generated 9 comments.
| File | Description |
|---|---|
| object/record.go | Adds driver-aware SQL expression helpers for timestamp conversion/formatting and updates metrics queries to use them. |
| casdoor/init.go | Adds JWKS fetching/parsing logic to dynamically load the JWT certificate at startup with fallback behavior. |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
| case "mssql": | ||
| return "DATEDIFF(SECOND, '1970-01-01', CONVERT(datetime, " + col + "))" | ||
| case "sqlite": |
There was a problem hiding this comment.
unixTimestampExpr() uses CONVERT(datetime, created_time) for MSSQL, but CreatedTime values are stored as RFC3339 strings (see util.GetCurrentTime()), which commonly include T and a timezone offset (e.g., +08:00) that datetime conversion may fail to parse. Use datetimeoffset parsing (e.g., CONVERT(datetimeoffset, col, 127)) and base DATEDIFF on that, and align the MSSQL branch in dateFormatExpr() similarly.
| if cert, err := fetchCertificateFromJWKS(jwksUrl); err == nil && cert != "" { | ||
| jwtPublicKey = cert | ||
| beego.Info("Successfully loaded JWT certificate from JWKS endpoint") | ||
| } else { | ||
| beego.Warn("Failed to fetch certificate from JWKS (", err, "), using embedded certificate") |
There was a problem hiding this comment.
The warning path logs "Failed to fetch certificate" even when err == nil but cert == "" (because the if condition also checks cert != ""). This makes logs misleading. Treat empty cert as an error inside fetchCertificateFromJWKS, or split the condition so the warn log accurately reflects whether there was an error vs. an empty/unsupported JWKS response.
| if cert, err := fetchCertificateFromJWKS(jwksUrl); err == nil && cert != "" { | |
| jwtPublicKey = cert | |
| beego.Info("Successfully loaded JWT certificate from JWKS endpoint") | |
| } else { | |
| beego.Warn("Failed to fetch certificate from JWKS (", err, "), using embedded certificate") | |
| if cert, err := fetchCertificateFromJWKS(jwksUrl); err != nil { | |
| beego.Warn("Failed to fetch certificate from JWKS (", err, "), using embedded certificate") | |
| } else if cert == "" { | |
| beego.Warn("JWKS endpoint returned empty certificate, using embedded certificate") | |
| } else { | |
| jwtPublicKey = cert | |
| beego.Info("Successfully loaded JWT certificate from JWKS endpoint") |
| if len(jwks.Keys) == 0 || len(jwks.Keys[0].X5C) == 0 { | ||
| return "", fmt.Errorf("no certificates in JWKS") | ||
| } | ||
|
|
||
| // Extract the first certificate and format as PEM | ||
| certData := jwks.Keys[0].X5C[0] |
There was a problem hiding this comment.
JWKS responses can contain multiple keys, and the first key may not have an x5c chain. The current implementation only inspects jwks.Keys[0] and will fail even if a later key contains a certificate. Iterate over jwks.Keys and select the first key that has a non-empty x5c (or match by kid if available).
| if len(jwks.Keys) == 0 || len(jwks.Keys[0].X5C) == 0 { | |
| return "", fmt.Errorf("no certificates in JWKS") | |
| } | |
| // Extract the first certificate and format as PEM | |
| certData := jwks.Keys[0].X5C[0] | |
| // Find the first key that has a non-empty X5C certificate chain | |
| var certData string | |
| for _, key := range jwks.Keys { | |
| if len(key.X5C) > 0 && key.X5C[0] != "" { | |
| certData = key.X5C[0] | |
| break | |
| } | |
| } | |
| if certData == "" { | |
| return "", fmt.Errorf("no certificates in JWKS") | |
| } | |
| // Extract the certificate and format as PEM |
| // Extract the first certificate and format as PEM | ||
| certData := jwks.Keys[0].X5C[0] | ||
| pemCert := fmt.Sprintf("-----BEGIN CERTIFICATE-----\n%s\n-----END CERTIFICATE-----\n", certData) | ||
|
|
||
| return pemCert, nil |
There was a problem hiding this comment.
The constructed PEM embeds the x5c value as a single long base64 line and does not validate that it decodes to a real X.509 cert. Some PEM parsers are strict about line wrapping, and returning an invalid PEM would silently break JWT verification. Consider base64-decoding x5c, validating via x509.ParseCertificate, then emitting PEM via pem.EncodeToMemory (which also handles line wrapping).
| func fetchCertificateFromJWKS(url string) (string, error) { | ||
| resp, err := http.Get(url) | ||
| if err != nil { | ||
| return "", err |
There was a problem hiding this comment.
fetchCertificateFromJWKS() uses http.Get without a timeout. Because this runs during InitCasdoorConfig(), a slow/hung network call can block application startup indefinitely. Use an http.Client with a reasonable timeout (and ideally a context-aware request) for this fetch.
| body, err := io.ReadAll(resp.Body) | ||
| if err != nil { | ||
| return "", err | ||
| } | ||
|
|
||
| var jwks JWKSResponse | ||
| if err := json.Unmarshal(body, &jwks); err != nil { | ||
| return "", err | ||
| } | ||
|
|
There was a problem hiding this comment.
io.ReadAll(resp.Body) reads the entire JWKS response unbounded. A large/malicious response could cause excessive memory usage during startup. Prefer json.NewDecoder(io.LimitReader(resp.Body, ...)) or otherwise cap the number of bytes read.
| body, err := io.ReadAll(resp.Body) | |
| if err != nil { | |
| return "", err | |
| } | |
| var jwks JWKSResponse | |
| if err := json.Unmarshal(body, &jwks); err != nil { | |
| return "", err | |
| } | |
| var jwks JWKSResponse | |
| decoder := json.NewDecoder(io.LimitReader(resp.Body, 1<<20)) // limit JWKS response to 1 MiB | |
| if err := decoder.Decode(&jwks); err != nil { | |
| return "", err | |
| } |
There was a problem hiding this comment.
LGTM, but in my opinion it is slightly excessive. It is not "random" service configured by the malicious user, but IdP provided in configuration by waf admin.
| func InitCasdoorConfig() { | ||
| casdoorEndpoint := beego.AppConfig.String("casdoorEndpoint") | ||
| clientId := beego.AppConfig.String("clientId") | ||
| clientSecret := beego.AppConfig.String("clientSecret") | ||
| casdoorOrganization := beego.AppConfig.String("casdoorOrganization") | ||
| casdoorApplication := beego.AppConfig.String("casdoorApplication") | ||
| casdoorsdk.InitConfig(casdoorEndpoint, clientId, clientSecret, JwtPublicKey, casdoorOrganization, casdoorApplication) | ||
|
|
||
| // Try to fetch certificate from Casdoor's standard JWKS endpoint | ||
| jwtPublicKey := JwtPublicKey // default to embedded certificate | ||
|
|
||
| jwksUrl := fmt.Sprintf("%s/.well-known/jwks", strings.TrimRight(casdoorEndpoint, "/")) | ||
| beego.Info("Fetching JWT certificate from JWKS: ", jwksUrl) | ||
|
|
||
| if cert, err := fetchCertificateFromJWKS(jwksUrl); err == nil && cert != "" { | ||
| jwtPublicKey = cert | ||
| beego.Info("Successfully loaded JWT certificate from JWKS endpoint") | ||
| } else { | ||
| beego.Warn("Failed to fetch certificate from JWKS (", err, "), using embedded certificate") | ||
| } | ||
|
|
||
| casdoorsdk.InitConfig(casdoorEndpoint, clientId, clientSecret, jwtPublicKey, casdoorOrganization, casdoorApplication) |
There was a problem hiding this comment.
InitCasdoorConfig() now performs an outbound HTTP request during startup. This function is invoked from tests (e.g., object/site_test.go, object/cert_test.go), so the test suite/CI will now depend on external network availability and Casdoor uptime, which can make runs flaky. Consider adding a config/env switch to disable JWKS fetching in tests/CI, or injecting the JWKS URL/HTTP client so tests can point to a local stub.
There was a problem hiding this comment.
The code still falls back to the embedded public key.
jwtPublicKey := JwtPublicKey // default to embedded certificate
|
hi @pawelmarkowski , you should only focus on one question in one issue/PR. this PR contains two questions. and you should create issue first before making PR. we need to run our community AI to draft the code first |
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
|
ok, I can close the PR then and I will split it into two |
This pull request improves database compatibility for time-based metrics queries and enhances Casdoor JWT certificate handling. The main changes include adding database-agnostic SQL helpers for timestamp formatting and conversion, and updating Casdoor initialization to fetch JWT certificates dynamically from a JWKS endpoint.
Database compatibility improvements:
unixTimestampExpranddateFormatExprhelper functions inobject/record.goto generate SQL expressions for converting and formatting timestamps, supporting MySQL, PostgreSQL, MSSQL, and SQLite. Metrics queries now use these helpers for cross-database compatibility. [1] [2]timeType2Formatto return driver-specific date/time format strings, ensuring correct grouping for metrics over time across different databases.Casdoor JWT certificate handling:
InitCasdoorConfigincasdoor/init.goto fetch the JWT public certificate from Casdoor's JWKS endpoint at runtime, falling back to the embedded certificate if fetching fails. This improves security and flexibility when validating JWTs. Added supporting types and thefetchCertificateFromJWKShelper function. [1] [2]