Packagist · contao/contao
Contao crawler leaks auth credentials to external hosts
Contao's crawler tries to prevent confidential HTTP client options from being sent to external domains by creating a scoped client: full options for root page origins, cleaned options for everything else. The cleaner removes Cookie and Authorization headers, but it removes the non-Symfony option names basic_auth and bearer_auth instead of Symfony HttpClient's real auth_basic and auth_bearer options.
When contao.crawl.default_http_client_options contains Basic or Bearer authentication for a protected staging/production site, those credentials remain in the "clean" client used for external links or configured additional URIs. An attacker who can get an external URL crawled, for example through a link on a crawled page while the broken-link checker is enabled, can receive the crawler credentials.
// core-bundle/src/Crawl/Escargot/Factory.php:175-209 @ e550b92a01ef625bd546e6c3956dd200af05ebf0
private function createHttpClient(array $options = []): HttpClientInterface
{
$options = array_merge_recursive(
[
'headers' => [
'accept' => 'text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8',
'user-agent' => self::USER_AGENT,
],
'max_duration' => 10,
],
array_merge_recursive($this->getDefaultHttpClientOptions(), $options),
);
$cleanOptions = $this->cleanOptionsFromConfidentialData($options);
if ($options === $cleanOptions) {
return ($this->httpClientFactory)($options);
}
$scopedOptionsByRegex = [];
foreach ($this->getRootPageUriCollection()->all() as $rootPageUri) {
$scopedOptionsByRegex[preg_quote($this->getOriginFromUri($rootPageUri))] = $options;
}
return new ScopingHttpClient(($this->httpClientFactory)($cleanOptions), $scopedOptionsByRegex);
}
// core-bundle/src/Crawl/Escargot/Factory.php:226-247 @ e550b92a01ef625bd546e6c3956dd200af05ebf0
foreach ($options as $k => $v) {
if ('headers' === $k) {
foreach ($v as $header => $value) {
if (\in_array(strtolower($header), ['authorization', 'cookie'], true)) {
continue;
}
$cleanOptions['headers'][$header] = $value;
}
continue;
}
if ('basic_auth' === $k || 'bearer_auth' === $k) {
continue;
}
$cleanOptions[$k] = $v;
}
Symfony HttpClient authentication options are auth_basic and auth_bearer; Contao's own manual documents auth_basic for crawler Basic Authentication. Because the cleaner only strips basic_auth and bearer_auth, the "clean" default client for non-root-page hosts still carries the real auth options. The existing factory test intends to assert that Authorization is not sent to www.foreign-domain.com, but its mock client factory ignores the $defaultOptions argument, so it does not catch auth options that survive into HttpClient::create($cleanOptions).
Strip the actual Symfony HttpClient authentication option keys from the clean client. Include NTLM as a defensive extension because Symfony documents it as another auth option.
- if ('basic_auth' === $k || 'bearer_auth' === $k) {
+ if (\in_array($k, ['auth_basic', 'auth_bearer', 'auth_ntlm', 'basic_auth', 'bearer_auth'], true)) {
continue;
}
Also update the factory test so the mock factory records or preserves $defaultOptions; otherwise the test does not verify what HttpClient::create($cleanOptions) receives in production.
https://attacker.example/probe on a page that the crawler visits. When an administrator or scheduled maintenance run starts the broken-link checker with crawler Basic/Bearer authentication configured, the request to the attacker URL includes the generated Authorization header.Is your project exposed to this? Stateward checks every dependency on every pull request and flags it only if your code actually reaches it.
Check my repoSources: CISA KEV (public domain), OSV.dev & GitHub Advisory Database (CC-BY-4.0), FIRST EPSS, NVD/CWE (public domain). Served live from the Stateward advisory database.