Skip to content

[Bug]: Severe N+1 query loop performance regression on SMB External Storage (files_external) in NC 33 #59656

@Erik-D-Bryer

Description

@Erik-D-Bryer

⚠️ This issue respects the following points: ⚠️

Bug description

There's a performance problem on SMB External Storage (files_external) in NC 33 when listing directory contents in the web app. File listings take several seconds or more. Navigating to a directory causes PHP to enters a blocking CPU loop (e.g. tms_utime=208 / 2.08s or much greater) executing thousands of individual database queries even though there are ~10 files in the directory. Listing the directory with smbclient takes 0.5 seconds but PHP takes several seconds (or more) doing individual SELECT queries on every file in the tree, including parent dirs. I have attached a stack trace.

slowtrace.log2.txt

Steps to reproduce

  1. Mount an SMB share in the admin menu for sytem-wide External Storage (Authentication: Log-in credentials, save in database).
  2. Ensure filesystem_check_changes is set to 1.
  3. Create a directory on the SMB share containing 1000 files (e.g. for ((i=1;i<=1000;i++)); do touch $i; done) and a subdir of that containing just a few files.
  4. Change dir to that subdir in the Nextcloud web app and wait for the few files to display.

Expected behavior

The few files in the subdir should display in 1 second or so.

Nextcloud Server version

33

Operating system

RHEL/CentOS

PHP engine version

PHP 8.3

Web server

Apache (supported)

Database engine version

MariaDB

Is this bug present after an update or on a fresh install?

Fresh Nextcloud Server install

Are you using the Nextcloud Server Encryption module?

Encryption is Disabled

What user-backends are you using?

  • Default user-backend (database)
  • LDAP/ Active Directory
  • SSO - SAML
  • Other

Configuration report

{
    "system": {
        "passwordsalt": "***REMOVED SENSITIVE VALUE***",
        "secret": "***REMOVED SENSITIVE VALUE***",
        "trusted_domains": [
            "localhost",
            "146.186.150.8",
            "data-cloud-test.hpc.psu.edu"
        ],
        "datadirectory": "***REMOVED SENSITIVE VALUE***",
        "dbtype": "mysql",
        "version": "33.0.2.2",
        "overwrite.cli.url": "https:\/\/data-cloud-test.hpc.psu.edu",
        "dbname": "***REMOVED SENSITIVE VALUE***",
        "dbhost": "***REMOVED SENSITIVE VALUE***",
        "dbtableprefix": "oc_",
        "mysql.utf8mb4": true,
        "dbuser": "***REMOVED SENSITIVE VALUE***",
        "dbpassword": "***REMOVED SENSITIVE VALUE***",
        "instanceid": "***REMOVED SENSITIVE VALUE***",
        "installed": true,
        "objectstore": {
            "class": "\\OC\\Files\\ObjectStore\\S3",
            "arguments": {
                "bucket": "data-cloud2",
                "key": "***REMOVED SENSITIVE VALUE***",
                "secret": "***REMOVED SENSITIVE VALUE***",
                "hostname": "vast2e-ip.hpc.psu.edu",
                "use_ssl": false,
                "use_path_style": true,
                "autocreate": false,
                "objectPrefix": "urn\/oid\/"
            }
        },
        "allow_user_to_change_display_name": false,
        "lost_password_link": "disabled",
        "oidc_login_provider_url": "https:\/\/login.microsoftonline.com\/7cf48d45-3ddb-4389-a9c1-c115526eb52e",
        "oidc_login_client_id": "128b419f-dff8-4688-8150-871e47c91e5f",
        "oidc_login_client_secret": "JJx8Q~zAa.Gin1kCtnW9Q3xUWAynEBzMwd~RUc3K",
        "oidc_login_button_text": "NOT WORKING: Log in with Penn State WebSSO",
        "oidc_login_hide_password_form": false,
        "oidc_login_attributes": {
            "id": "sub",
            "ldap_uid": "upn"
        },
        "oidc_login_scope": "openid profile",
        "oidc_login_proxy_ldap": true,
        "oidc_login_disable_registration": true,
        "oidc_login_tls_verify": false,
        "oidc_create_groups": false,
        "ldapProviderFactory": "OCA\\User_LDAP\\LDAPProviderFactory",
        "overwriteprotocol": "https",
        "session_lifetime": "86400",
        "session_keepalive": false,
        "auto_logout": true,
        "remember_login_cookie_lifetime": "86400",
        "mail_sendmailmode": "smtp",
        "maintenance_window_start": 0,
        "default_phone_region": "US",
        "filelocking.enabled": false,
        "memcache.locking": "\\OC\\Memcache\\Redis",
        "memcache.local": "\\OC\\Memcache\\Redis",
        "memcache.distributed": "\\OC\\Memcache\\Redis",
        "redis": {
            "host": "***REMOVED SENSITIVE VALUE***",
            "port": 0,
            "dbindex": 0,
            "timeout": "1.5"
        },
        "enable_file_metadata_collection": false,
        "loglevel": 2,
        "enable_previews": false,
        "dbport": ""
    }
}

List of activated Apps

Enabled:
  - activity: 6.0.0
  - app_api: 33.0.0
  - bruteforcesettings: 6.0.0
  - circles: 33.0.0
  - cloud_federation_api: 1.17.0
  - comments: 1.23.0
  - contactsinteraction: 1.14.1
  - dashboard: 7.13.0
  - dav: 1.36.0
  - federatedfilesharing: 1.23.0
  - federation: 1.23.0
  - files: 2.5.0
  - files_downloadlimit: 5.1.0
  - files_external: 1.25.1
  - files_pdfviewer: 6.0.0
  - files_reminders: 1.6.0
  - files_sharing: 1.25.2
  - files_trashbin: 1.23.0
  - files_versions: 1.26.0
  - firstrunwizard: 6.0.0
  - logreader: 6.0.0
  - lookup_server_connector: 1.21.0
  - nextcloud_announcements: 5.0.0
  - notifications: 6.0.0
  - oauth2: 1.21.0
  - password_policy: 5.0.0
  - privacy: 5.0.0
  - profile: 1.2.0
  - provisioning_api: 1.23.0
  - recommendations: 6.0.0
  - related_resources: 4.0.0
  - serverinfo: 5.0.0
  - settings: 1.16.0
  - sharebymail: 1.23.0
  - support: 5.0.0
  - survey_client: 5.0.0
  - systemtags: 1.23.0
  - text: 7.0.0
  - theming: 2.8.0
  - twofactor_backupcodes: 1.22.0
  - twofactor_totp: 15.0.0
  - updatenotification: 1.23.0
  - user_ldap: 1.24.0
  - user_status: 1.13.0
  - viewer: 6.0.0
  - weather_status: 1.13.0
  - webhook_listeners: 1.5.0
  - workflowengine: 2.15.0
Disabled:
  - admin_audit: 1.23.0
  - encryption: 2.21.0
  - photos: 6.0.0 (installed 6.0.0)
  - suspicious_login: 11.0.0
  - testing: 1.23.0
  - twofactor_nextcloud_notification: 7.0.0

Nextcloud Signing status

Technical information
=====================
The following list covers which files have failed the integrity check. Please read
the previous linked documentation to learn more about the errors and how to fix
them.

Results
=======
- core
	- EXTRA_FILE
		- core/img/favicon-psu.ico
		- core/img/logo/logo-psu-icds.png

Raw output
==========
Array
(
    [core] => Array
        (
            [EXTRA_FILE] => Array
                (
                    [core/img/favicon-psu.ico] => Array
                        (
                            [expected] => 
                            [current] => 658890bec722ee312c27b5e098c35cd6c249670754bb327e43180a5aec6f391dae7dbb51c904ae5c23e021a4a995d31d9f052a70d8b14805eb8889790bd2b1b0
                        )

                    [core/img/logo/logo-psu-icds.png] => Array
                        (
                            [expected] => 
                            [current] => 2404f83a9ab8cdc62fada6eacaa8250bef3c89445a36bf446bac98f572e35eb2cf65682102263992a6f0d38150a54edef14a3e56fb330e7b0ed18d639b93c43a
                        )

                )

        )

)

Nextcloud Logs

There are no nextcloud.log entries produced when triggering this bug.

Additional info

slowtrace.log2.txt
The attached is a stack trace of php-fpm (the only php-fpm process running during the test) while trying to browse a directory over SMB. It takes several seconds to list the files despite the directory having ~10 files in it. I believe this shows nextcloud doing a mariadb db query for each file in the file path tree, including parent dirs, which means many more lookups than are required. Browsing a directory triggers thousands of database lookups with SELECT filecache.fileid, storage, path, path_hash, filecache.parent, filecache.name, mimetype, mimepart, size, mtime, storage_mtime, encrypted, filecache.etag, filecache.permissions, checksum, unencrypted_size FROM oc_filecache... that together consume several seconds. The files_external SMB wrapper does not seem to make a batch query and instead queries the database individually for every single file. The stack trace was generated for a browse attempt on one of the faster file systems, taking a few seconds. File systems with more directories at the parent level took much longer.

[ezb5568@data-cloud-test nextcloud]$ rpm -qa |grep smb
libsmbclient-4.19.4-15.el8_10.x86_64
php-smbclient-1.1.2-1.el8.remi.8.3.x86_64

[ezb5568@data-cloud-test nextcloud]$ rpm -qa |grep php
php-opcache-8.3.30-1.module_php.8.3.el8.remi.x86_64
php-mysqlnd-8.3.30-1.module_php.8.3.el8.remi.x86_64
php-cli-8.3.30-1.module_php.8.3.el8.remi.x86_64
gd3php-2.3.3-8.el8.remi.x86_64
php-pecl-imagick-im7-3.8.1-1.module_php.8.3.el8.remi.x86_64
php-ldap-8.3.30-1.module_php.8.3.el8.remi.x86_64
php-common-8.3.30-1.module_php.8.3.el8.remi.x86_64
php-mbstring-8.3.30-1.module_php.8.3.el8.remi.x86_64
php-pecl-igbinary-3.2.16-1.el8.remi.8.3.x86_64
php-8.3.30-1.module_php.8.3.el8.remi.x86_64
php-smbclient-1.1.2-1.el8.remi.8.3.x86_64
php-gmp-8.3.30-1.module_php.8.3.el8.remi.x86_64
php-xml-8.3.30-1.module_php.8.3.el8.remi.x86_64
php-pdo-8.3.30-1.module_php.8.3.el8.remi.x86_64
php-sodium-8.3.30-1.module_php.8.3.el8.remi.x86_64
php-pecl-redis6-6.3.0-1.module_php.8.3.el8.remi.x86_64
php-gd-8.3.30-1.module_php.8.3.el8.remi.x86_64
php-process-8.3.30-1.module_php.8.3.el8.remi.x86_64
php-bcmath-8.3.30-1.module_php.8.3.el8.remi.x86_64
php-fpm-8.3.30-1.module_php.8.3.el8.remi.x86_64
php-intl-8.3.30-1.module_php.8.3.el8.remi.x86_64
php-pecl-apcu-5.1.28-1.module_php.8.3.el8.remi.x86_64
oniguruma5php-6.9.10-1.el8.remi.x86_64
php-pecl-msgpack-3.0.0-1.el8.remi.8.3.x86_64
php-pecl-zip-1.22.8-1.module_php.8.3.el8.remi.x86_64

[ezb5568@data-cloud-test nextcloud]$ uname -a
Linux data-cloud-test 4.18.0-553.111.1.el8_10.x86_64 #1 SMP Sat Feb 28 00:49:56 EST 2026 x86_64 x86_64 x86_64 GNU/Linux

[ezb5568@data-cloud-test nextcloud]$ rpm -qa | grep httpd
httpd-filesystem-2.4.37-65.module+el8.10.0+23815+1b5e1c66.7.noarch
httpd-2.4.37-65.module+el8.10.0+23815+1b5e1c66.7.x86_64
httpd-tools-2.4.37-65.module+el8.10.0+23815+1b5e1c66.7.x86_64

[ezb5568@data-cloud-test nextcloud]$ rpm -qa | grep -i mariadb
mariadb-gssapi-server-10.3.39-2.module+el8.10.0+23756+5eac0e54.x86_64
mariadb-connector-c-config-3.1.11-2.el8_3.noarch
mariadb-errmsg-10.3.39-2.module+el8.10.0+23756+5eac0e54.x86_64
mariadb-connector-c-devel-3.1.11-2.el8_3.x86_64
mariadb-devel-10.3.39-2.module+el8.10.0+23756+5eac0e54.x86_64
mariadb-10.3.39-2.module+el8.10.0+23756+5eac0e54.x86_64
mariadb-server-10.3.39-2.module+el8.10.0+23756+5eac0e54.x86_64
mariadb-connector-c-3.1.11-2.el8_3.x86_64
mariadb-common-10.3.39-2.module+el8.10.0+23756+5eac0e54.x86_64
mariadb-backup-10.3.39-2.module+el8.10.0+23756+5eac0e54.x86_64
mariadb-server-utils-10.3.39-2.module+el8.10.0+23756+5eac0e54.x86_64

Metadata

Metadata

Assignees

No one assigned

    Labels

    0. Needs triagePending check for reproducibility or if it fits our roadmap33-feedbackbug

    Type

    No fields configured for Bug.

    Projects

    Status

    To triage

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions