Why this matters beyond the homelab: Multi-component security platforms frequently carry more than one identity store, and the tooling rarely makes that obvious. Credential drift between them produces failures that look like network problems, certificate problems, or version problems, and you can burn hours in the wrong subsystem before realizing you are holding two passwords for what you assumed was one account.
The Symptom
The Wazuh dashboard loaded fine. I could log in. Agents were reporting, alerts were flowing into the indexer, data was being collected normally.
But the API connection indicator showed offline, and every part of the interface that queries the manager API rather than the indexer was dead: agent management, configuration views, active response controls.
Data collection working while management is broken is a useful clue in itself. It means the pipeline into storage is healthy and the control plane is not, which immediately splits the system into two halves and tells you which one to investigate.
The Architecture Nobody Explains Up Front
Wazuh is three components, and this is the part worth internalizing:
The manager collects from agents, runs the rules engine, and exposes a management API on port 55000.
The indexer is an OpenSearch fork that stores alerts.
The dashboard is an OpenSearch Dashboards fork. It queries the indexer for data and the manager API for control operations.
Those last two connections use completely separate authentication systems:
Indexer users live in OpenSearch’s security plugin, defined in internal_users.yml, managed with the bundled password tool.
Manager API users live in a separate RBAC store, a SQLite database at /var/ossec/api/configuration/security/rbac.db.
Different backends. Different management tooling. No synchronization between them.
The trap is right here:
/usr/share/wazuh-indexer/plugins/opensearch-security/tools/wazuh-passwords-tool.sh
Given its name and its prominence in the documentation, this reads as the password tool for Wazuh. It is not. It only manages OpenSearch indexer users. It does not touch the manager API’s RBAC store at all.
So you can run the official password tool, see it succeed, and have changed nothing about the credential the dashboard uses to reach the manager API.
What Had Actually Happened
Months earlier, during unrelated work, I had reset the API user’s password by writing directly to the RBAC SQLite database. That was necessary at the time because the stored credential was invalid and the normal reset path was not available.
The reset worked. The manager API accepted the new password.
The dashboard’s configuration file still held the original install-time credential, a long random string generated at installation. Nothing had updated it, because nothing knew it needed updating.
The dashboard had been authenticating against the manager API with a stale password ever since. The API rejected it, the dashboard reported the connection offline, and the indexer side kept working perfectly because it uses a different credential entirely.
Months of a half-broken security platform, from one password reset with no propagation step.
Diagnosis
Confirm which credential the API actually accepts, by authenticating directly and bypassing the dashboard completely:
curl -u wazuh-wui:'<candidate-password>' -k -X POST \
"https://<manager-ip>:55000/security/user/authenticate?raw=true"
A JWT comes back if the credential is right. An error if it is not. This is the single most valuable command in the whole exercise, because it isolates the question to “does this password work” with no other moving parts.
Then find what the dashboard is configured to send:
grep -n "password" /usr/share/wazuh-dashboard/data/wazuh/config/wazuh.yml
In my case the answer was line 225, holding the install-time string. Comparing that against the password that had just produced a working JWT made the drift obvious.
Then correct it and restart:
systemctl restart wazuh-dashboard
Connection restored.
Why This Class of Bug Is Expensive
An offline API connection presents like an infrastructure problem. The instinct is to check the network path, verify the manager is listening on 55000, look at certificates, check firewall rules between the dashboard and manager, look for a version mismatch.
All of those are reasonable and all of them are wrong here. The connection was fine. The credential was wrong. But nothing in the symptom points at credentials, because you are not thinking of the dashboard as a client that authenticates. It feels like one system talking to itself.
The general lesson: when a component of a platform cannot reach another component of the same platform, check whether that is an authenticated connection before assuming it is a transport problem. Internal service-to-service auth is easy to forget precisely because it is internal.
What I Changed
Documented both auth systems explicitly, in the runbook, with which tool manages which store and which config files hold which credentials. The information exists in Wazuh’s documentation but is not surfaced where you need it, which is at the moment you are resetting a password.
Made propagation part of the reset procedure. Changing a service credential is never one step. It is: change it, find every consumer, update each one, restart each one, verify each one. The reset is the easy part.
Put all four credentials in the vault, distinctly labeled by which store they belong to. Two systems with similar-sounding usernames is exactly where you paste the wrong value.
Added a synthetic check on the API connection. A monitor that authenticates to the manager API on a schedule and alerts on failure. This exact problem would have been caught in minutes rather than months, and the check is a handful of lines.
Takeaways
Wazuh has two independent identity stores. OpenSearch users in internal_users.yml, API users in the RBAC SQLite database.
The bundled password tool only covers one of them. Its name suggests otherwise.
Direct database credential edits do not propagate. Nothing downstream is notified.
Isolate credential questions with a direct auth call. curl against the API answers in one command what the dashboard cannot tell you.
Internal service-to-service connections are authenticated too. Check credentials before chasing the network.
Credential changes are a procedure, not an action. Change, propagate, restart, verify.
Monitor internal API connections synthetically. Otherwise a broken control plane sits behind a working data plane indefinitely.
Related: Deploying Fleet-Wide SIEM Across a Proxmox Cluster | Rules That Match Nothing | Vaultwarden: Self-Hosted Password Management