Repository navigation
Closed
Labels
0. Needs triagePending check for reproducibility or if it fits our roadmapPending check for reproducibility or if it fits our roadmap26-feedbackbug
Description
⚠️ This issue respects the following points: ⚠️
- This is a bug, not a question or a configuration/webserver/proxy issue.
- This issue is not already reported on Github (I've searched it).
- Nextcloud Server is up to date. See Maintenance and Release Schedule for supported versions.
- Nextcloud Server is running on 64bit capable CPU, PHP and OS.
- I agree to follow Nextcloud's Code of Conduct.
Bug description
When downloading largeish folders (>90 files, subfolders, etc.) on MacOS, Nextcloud fails to create the tar file of the folder.
Nextcloud uses the TarStreamer library here:
https://github.com/owncloud/TarStreamer
But it is apparently not compatible with current versions of PHP:
owncloud/TarStreamer#23 (comment)
The library needs to be brought up to date in order to use Nextcloud with secure versions of PHP/MacOS.
Steps to reproduce
- Create a folder with 90 files in a subfolder
- Open a browser on MacOS
- Attempt to download the folder
- Receive a truncated tar file
Expected behavior
Receive a complete tar file of the folder.
Installation method
Community Manual installation with Archive
Nextcloud Server version
26
Operating system
Debian/Ubuntu
PHP engine version
PHP 8.1
Web server
Apache (supported)
Database engine version
MariaDB
Is this bug present after an update or on a fresh install?
Updated from a minor version (ex. 22.2.3 to 22.2.4)
Are you using the Nextcloud Server Encryption module?
Encryption is Disabled
What user-backends are you using?
- Default user-backend (database)
- LDAP/ Active Directory
- SSO - SAML
- Other
Configuration report
{
"system": {
"instanceid": "***REMOVED SENSITIVE VALUE***",
"passwordsalt": "***REMOVED SENSITIVE VALUE***",
"secret": "***REMOVED SENSITIVE VALUE***",
"trusted_domains": [
"10.150.64.10",
"***REMOVED SENSITIVE VALUE***"
],
"datadirectory": "***REMOVED SENSITIVE VALUE***",
"dbtype": "mysql",
"version": "26.0.0.11",
"overwrite.cli.url": "***REMOVED SENSITIVE VALUE***",
"htaccess.RewriteBase": "\/",
"dbname": "***REMOVED SENSITIVE VALUE***",
"dbhost": "***REMOVED SENSITIVE VALUE***",
"dbport": "",
"dbtableprefix": "oc_",
"mysql.utf8mb4": true,
"dbuser": "***REMOVED SENSITIVE VALUE***",
"dbpassword": "***REMOVED SENSITIVE VALUE***",
"installed": true,
"skeletondirectory": "\/var\/www\/skeleton",
"trashbin_retention_obligation": "1, 2",
"ldapUserCleanupInterval": "30",
"log_type": "syslog",
"logfile": "",
"loglevel": 1,
"syslog_tag": "Nextcloud",
"enable_previews": false,
"log.condition": {
"apps": [
"admin_audit"
]
},
"default_phone_region": "CA",
"memcache.local": "\\OC\\Memcache\\APCu",
"filelocking.enabled": "true",
"memcache.locking": "\\OC\\Memcache\\Redis",
"redis": {
"host": "***REMOVED SENSITIVE VALUE***",
"port": 0,
"dbindex": 0,
"password": "***REMOVED SENSITIVE VALUE***",
"timeout": 1.5
},
"maintenance": false,
"ldapProviderFactory": "OCA\\User_LDAP\\LDAPProviderFactory"
}
}List of activated Apps
Enabled:
- activity: 2.18.0
- admin_audit: 1.16.0
- bruteforcesettings: 2.6.0
- cloud_federation_api: 1.9.0
- contacts: 5.2.0
- dav: 1.25.0
- federatedfilesharing: 1.16.0
- files: 1.21.1
- files_downloadactivity: 1.16.0
- files_external: 1.18.0
- files_pdfviewer: 2.7.0
- files_rightclick: 1.5.0
- files_sharing: 1.18.0
- files_trashbin: 1.16.0
- lookup_server_connector: 1.14.0
- oauth2: 1.14.0
- password_policy: 1.16.0
- privacy: 1.10.0
- provisioning_api: 1.16.0
- related_resources: 1.1.0-alpha1
- serverinfo: 1.16.0
- settings: 1.8.0
- text: 3.7.2
- theming: 2.1.1
- twofactor_backupcodes: 1.15.0
- user_ldap: 1.16.0
- viewer: 1.10.0
- workflowengine: 2.8.0
Disabled:
- circles: 26.0.0 (installed 22.1.1)
- comments: 1.16.0 (installed 1.12.0)
- contactsinteraction: 1.7.0 (installed 1.3.0)
- dashboard: 7.6.0 (installed 7.2.0)
- encryption: 2.14.0
- federation: 1.16.0 (installed 1.12.0)
- files_versions: 1.19.1 (installed 1.15.0)
- firstrunwizard: 2.15.0 (installed 2.11.0)
- group_everyone: 0.1.11 (installed 0.1.11)
- logreader: 2.11.0 (installed 2.7.0)
- nextcloud_announcements: 1.15.0 (installed 1.11.0)
- notifications: 2.14.0 (installed 2.10.1)
- photos: 2.2.0 (installed 1.4.0)
- recommendations: 1.5.0 (installed 1.1.0)
- sharebymail: 1.16.0 (installed 1.12.0)
- support: 1.9.0 (installed 1.5.0)
- survey_client: 1.14.0 (installed 1.10.0)
- suspicious_login: 4.4.0
- systemtags: 1.16.0 (installed 1.12.0)
- twofactor_totp: 8.0.0-alpha.0
- updatenotification: 1.16.0 (installed 1.15.0)
- user_status: 1.6.0 (installed 1.2.0)
- weather_status: 1.6.0 (installed 1.2.0)Nextcloud Signing status
No errors have been found.Nextcloud Logs
Relevant log lines:
nc_msg_Exception
TypeError
nc_msg_File
/var/www/nextcloud/3rdparty/deepdiver1975/tarstreamer/src/TarHeader.php
nc_msg_Line
72
nc_msg_Message
decoct(): Argument #1 ($num) must be of type int, string given
nc_msg_Trace
{file=/var/www/nextcloud/3rdparty/deepdiver1975/tarstreamer/src/TarHeader.php, line=72, function=decoct},
{file=/var/www/nextcloud/3rdparty/deepdiver1975/tarstreamer/src/TarStreamer.php, line=200, function=getHeader, class=ownCloud\TarStreamer\TarHeader, type=->},
{file=/var/www/nextcloud/3rdparty/deepdiver1975/tarstreamer/src/TarStreamer.php, line=166, function=writeLongName, class=ownCloud\TarStreamer\TarStreamer, type=->},
{file=/var/www/nextcloud/3rdparty/deepdiver1975/tarstreamer/src/TarStreamer.php, line=102, function=initFileStreamTransfer, class=ownCloud\TarStreamer\TarStreamer, type=->},
{file=/var/www/nextcloud/lib/private/Streamer.php, line=167, function=addFileFromStream, class=ownCloud\TarStreamer\TarStreamer, type=->},
{file=/var/www/nextcloud/lib/private/Streamer.php, line=136, function=addFileFromStream, class=OC\Streamer, type=->},
{file=/var/www/nextcloud/lib/private/Streamer.php, line=141, function=addDirRecursive, class=OC\Streamer, type=->},
{file=/var/www/nextcloud/lib/private/legacy/OC_Files.php, line=217, function=addDirRecursive, class=OC\Streamer, type=->},
{file=/var/www/nextcloud/apps/files/ajax/download.php, line=77, function=get, class=OC_Files, type=::},
{file=/var/www/nextcloud/lib/private/Route/Route.php, line=155, args=[/var/www/nextcloud/apps/files/ajax/download.php], function=require_once},
{function=OC\Route\{closure}, class=OC\Route\Route, type=->, args=[*** sensitive parameters replaced ***]},
{file=/var/www/nextcloud/lib/private/Route/Router.php, line=324, function=call_user_func},
{file=/var/www/nextcloud/lib/base.php, line=1055, function=match, class=OC\Route\Router, type=->},
{file=/var/www/nextcloud/index.php, line=36, function=handleRequest, class=OC, type=::}
Full Logs:
application_name
Nextcloud
facility
user-level
facility_num
1
level
3
message
{"reqId":"DxUGPZDJ1qX1WR8O81ll","level":3,"time":"2023-04-14T20:21:01+00:00","remoteAddr":"10.150.64.1","user":"***REMOVED SENSITIVE VALUE***","app":"index","method":"GET","url":"/index.php/apps/files/ajax/download.php?***REMOVED SENSITIVE VALUE***","message":"{\"Exception\":\"TypeError\",\"Message\":\"decoct(): Argument #1 ($num) must be of type int, string given\",\"Code\":0,\"Trace\":[{\"file\":\"/var/www/nextcloud/3rdparty/deepdiver1975/tarstreamer/src/TarHeader.php\",\"line\":72,\"function\":\"decoct\"},{\"file\":\"/var/www/nextcloud/3rdparty/deepdiver1975/tarstreamer/src/TarStreamer.php\",\"line\":200,\"function\":\"getHeader\",\"class\":\"ownCloud\\\\TarStreamer\\\\TarHeader\",\"type\":\"->\"},{\"file\":\"/var/www/nextcloud/3rdparty/deepdiver1975/tarstreamer/src/TarStreamer.php\",\"line\":166,\"function\":\"writeLongName\",\"class\":\"ownCloud\\\\TarStreamer\\\\TarStreamer\",\"type\":\"->\"},{\"file\":\"/var/www/nextcloud/3rdparty/deepdiver1975/tarstreamer/src/TarStreamer.php\",\"line\":102,\"function\":\"initFileStreamTransfer\",\"class\":\"ownCloud\\\\TarStreamer\\\\TarStreamer\",\"type\":\"->\"},{\"file\":\"/var/www/nextcloud/lib/private/Streamer.php\",\"line\":167,\"function\":\"addFileFromStream\",\"class\":\"ownCloud\\\\TarStreamer\\\\TarStreamer\",\"type\":\"->\"},{\"file\":\"/var/www/nextcloud/lib/private/Streamer.php\",\"line\":136,\"function\":\"addFileFromStream\",\"class\":\"OC\\\\Streamer\",\"type\":\"->\"},{\"file\":\"/var/www/nextcloud/lib/private/Streamer.php\",\"line\":141,\"function\":\"addDirRecursive\",\"class\":\"OC\\\\Streamer\",\"type\":\"->\"},{\"file\":\"/var/www/nextcloud/lib/private/legacy/OC_Files.php\",\"line\":217,\"function\":\"addDirRecursive\",\"class\":\"OC\\\\Streamer\",\"type\":\"->\"},{\"file\":\"/var/www/nextcloud/apps/files/ajax/download.php\",\"line\":77,\"function\":\"get\",\"class\":\"OC_Files\",\"type\":\"::\"},{\"file\":\"/var/www/nextcloud/lib/private/Route/Route.php\",\"line\":155,\"args\":[\"/var/www/nextcloud/apps/files/ajax/download.php\"],\"function\":\"require_once\"},{\"function\":\"OC\\\\Route\\\\{closure}\",\"class\":\"OC\\\\Route\\\\Route\",\"type\":\"->\",\"args\":[\"*** sensitive parameters replaced ***\"]},{\"file\":\"/var/www/nextcloud/lib/private/Route/Router.php\",\"line\":324,\"function\":\"call_user_func\"},{\"file\":\"/var/www/nextcloud/lib/base.php\",\"line\":1055,\"function\":\"match\",\"class\":\"OC\\\\Route\\\\Router\",\"type\":\"->\"},{\"file\":\"/var/www/nextcloud/index.php\",\"line\":36,\"function\":\"handleRequest\",\"class\":\"OC\",\"type\":\"::\"}],\"File\":\"/var/www/nextcloud/3rdparty/deepdiver1975/tarstreamer/src/TarHeader.php\",\"Line\":72,\"CustomMessage\":\"--\"}","userAgent":"Mozilla/5.0 (Macintosh; Intel Mac OS X 10.15; rv:109.0) Gecko/20100101 Firefox/109.0","version":"26.0.0.11"}
nc_app
index
nc_level
3
nc_message
{"Exception":"TypeError","Message":"decoct(): Argument #1 ($num) must be of type int, string given","Code":0,"Trace":[{"file":"/var/www/nextcloud/3rdparty/deepdiver1975/tarstreamer/src/TarHeader.php","line":72,"function":"decoct"},{"file":"/var/www/nextcloud/3rdparty/deepdiver1975/tarstreamer/src/TarStreamer.php","line":200,"function":"getHeader","class":"ownCloud\\TarStreamer\\TarHeader","type":"->"},{"file":"/var/www/nextcloud/3rdparty/deepdiver1975/tarstreamer/src/TarStreamer.php","line":166,"function":"writeLongName","class":"ownCloud\\TarStreamer\\TarStreamer","type":"->"},{"file":"/var/www/nextcloud/3rdparty/deepdiver1975/tarstreamer/src/TarStreamer.php","line":102,"function":"initFileStreamTransfer","class":"ownCloud\\TarStreamer\\TarStreamer","type":"->"},{"file":"/var/www/nextcloud/lib/private/Streamer.php","line":167,"function":"addFileFromStream","class":"ownCloud\\TarStreamer\\TarStreamer","type":"->"},{"file":"/var/www/nextcloud/lib/private/Streamer.php","line":136,"function":"addFileFromStream","class":"OC\\Streamer","type":"->"},{"file":"/var/www/nextcloud/lib/private/Streamer.php","line":141,"function":"addDirRecursive","class":"OC\\Streamer","type":"->"},{"file":"/var/www/nextcloud/lib/private/legacy/OC_Files.php","line":217,"function":"addDirRecursive","class":"OC\\Streamer","type":"->"},{"file":"/var/www/nextcloud/apps/files/ajax/download.php","line":77,"function":"get","class":"OC_Files","type":"::"},{"file":"/var/www/nextcloud/lib/private/Route/Route.php","line":155,"args":["/var/www/nextcloud/apps/files/ajax/download.php"],"function":"require_once"},{"function":"OC\\Route\\{closure}","class":"OC\\Route\\Route","type":"->","args":["*** sensitive parameters replaced ***"]},{"file":"/var/www/nextcloud/lib/private/Route/Router.php","line":324,"function":"call_user_func"},{"file":"/var/www/nextcloud/lib/base.php","line":1055,"function":"match","class":"OC\\Route\\Router","type":"->"},{"file":"/var/www/nextcloud/index.php","line":36,"function":"handleRequest","class":"OC","type":"::"}],"File":"/var/www/nextcloud/3rdparty/deepdiver1975/tarstreamer/src/TarHeader.php","Line":72,"CustomMessage":"--"}
nc_method
GET
nc_msg_Code
0
nc_msg_CustomMessage
--
nc_msg_Exception
TypeError
nc_msg_File
/var/www/nextcloud/3rdparty/deepdiver1975/tarstreamer/src/TarHeader.php
nc_msg_Line
72
nc_msg_Message
decoct(): Argument #1 ($num) must be of type int, string given
nc_msg_Trace
{file=/var/www/nextcloud/3rdparty/deepdiver1975/tarstreamer/src/TarHeader.php, line=72, function=decoct},
{file=/var/www/nextcloud/3rdparty/deepdiver1975/tarstreamer/src/TarStreamer.php, line=200, function=getHeader, class=ownCloud\TarStreamer\TarHeader, type=->},
{file=/var/www/nextcloud/3rdparty/deepdiver1975/tarstreamer/src/TarStreamer.php, line=166, function=writeLongName, class=ownCloud\TarStreamer\TarStreamer, type=->},
{file=/var/www/nextcloud/3rdparty/deepdiver1975/tarstreamer/src/TarStreamer.php, line=102, function=initFileStreamTransfer, class=ownCloud\TarStreamer\TarStreamer, type=->},
{file=/var/www/nextcloud/lib/private/Streamer.php, line=167, function=addFileFromStream, class=ownCloud\TarStreamer\TarStreamer, type=->},
{file=/var/www/nextcloud/lib/private/Streamer.php, line=136, function=addFileFromStream, class=OC\Streamer, type=->},
{file=/var/www/nextcloud/lib/private/Streamer.php, line=141, function=addDirRecursive, class=OC\Streamer, type=->},
{file=/var/www/nextcloud/lib/private/legacy/OC_Files.php, line=217, function=addDirRecursive, class=OC\Streamer, type=->},
{file=/var/www/nextcloud/apps/files/ajax/download.php, line=77, function=get, class=OC_Files, type=::},
{file=/var/www/nextcloud/lib/private/Route/Route.php, line=155, args=[/var/www/nextcloud/apps/files/ajax/download.php], function=require_once},
{function=OC\Route\{closure}, class=OC\Route\Route, type=->, args=[*** sensitive parameters replaced ***]},
{file=/var/www/nextcloud/lib/private/Route/Router.php, line=324, function=call_user_func},
{file=/var/www/nextcloud/lib/base.php, line=1055, function=match, class=OC\Route\Router, type=->},
{file=/var/www/nextcloud/index.php, line=36, function=handleRequest, class=OC, type=::}
nc_remoteAddr
10.150.64.1
nc_reqId
DxUGPZDJ1qX1WR8O81ll
nc_time
2023-04-14 16:21:01.000
nc_url
/index.php/apps/files/ajax/download.php?***REMOVED SENSITIVE VALUE***
nc_user
***REMOVED SENSITIVE VALUE***
nc_userAgent
Mozilla/5.0 (Macintosh; Intel Mac OS X 10.15; rv:109.0) Gecko/20100101 Firefox/109.0
nc_version
26.0.0.11
process_id
1822
source
nc
timestamp
2023-04-14 16:21:01.331Additional info
No response
Metadata
Metadata
Assignees
Labels
0. Needs triagePending check for reproducibility or if it fits our roadmapPending check for reproducibility or if it fits our roadmap26-feedbackbug
Activity
pjft commentedon Jun 7, 2023
I get this as well on Linux. Tried to download 5GB of multiple folders with smaller files, and the tar file comes back with 257MB only.
If it's downloaded as a ZIP file (i.e. smaller download), it works as intended. The moment it crosses the threshold and it becomes a TAR file, the TAR file gets downloaded, but it's incomplete.I cannot replicate it consistently, though, but will happily troubleshoot this further. It might be related to the size of the files it's archiving? I'll look into this and try to come up with an example that's replicable.This is the error:
pjft commentedon Jun 7, 2023
After further investigation, I am not convinced that this has anything to do with a PHP version incompatibility - at least not directly. The error does not happen all the time - I'm just finishing downloading a 4GB TAR archive and all is well.
EDIT: and, on the flipside, I can consistently replicate this with a specific folder, particularly in a file that does not get added.
It crashes when decoding the mtime of the file being compressed, so I will look into that and report back in the coming weeks.
pjft commentedon Jun 13, 2023
In my case, this is fixed with the following change:
In
/var/www/html/nextcloud/3rdparty/deepdiver1975/tarstreamer/src/TarHeader.phpChange:
To:
to force the explicit int cast.
@szaimen since you tagged this issue, I'd appreciate guidance here. I'm happy to submit a PR, but should I do it upstream, assuming they'll take it, or do you have any other recommendation?
Thanks all.
szaimen commentedon Jun 13, 2023
Best to open a PR here then: https://github.com/owncloud/TarStreamer
also cc @nextcloud/server-backend
pjft commentedon Jun 13, 2023
Thank you. I submitted the PR here:
owncloud/TarStreamer#25
I'll keep you posted.
pjft commentedon Jun 14, 2023
Thanks for the guidance - the PR has been approved.
Is there anything that needs to be done on the Nextcloud front (a PR or something) to incorporate those changes here?
szaimen commentedon Jun 14, 2023
I guess they would need to publish a new release. Afterwards we can update the dependency here.
come-nc commentedon Jun 15, 2023
The cast to int will fail on 32bit for files larger than 4GB, not sure if this is a big deal or not (and whether without the cast it will work or not on 32bits).
-> I tested and it seems even without the cast this will crash on 32bit for sizes above 4GB, so the PR does not break anything.
pjft commentedon Jun 15, 2023
@come-nc Thanks for the heads up.
Just to make sure I'm reading this correctly, what you're saying is that TarStreamer fails on 32bit systems on sizes above 4GB, with or without the patch. Is that correct?
There's a lot of nuances on 32bit systems and 4GB file sizes indeed. Well spotted.
come-nc commentedon Jun 15, 2023
Yes I think so, because those sizes on 32bits will always appears as float and decoct expects an int.
gotton commentedon Jun 28, 2023
Hello.
I can confirm i got this issue with 64bits system.
I can also confirm i have solved it with the proposal from @pjft ont TarHeader.
Also on mac with firefox.
szaimen commentedon Jun 28, 2023
Apparently there is a new release already: https://packagist.org/packages/owncloud/tarstreamer#v2.1.0
pjft commentedon Jun 28, 2023
Thanks for pushing for that. I take it that, come the next Nextcloud update, it will include this updated package, is that it? Or is there a separate update process?
Best.
szaimen commentedon Jun 28, 2023
There is. Cc @nextcloud/server-backend
pjft commentedon Jul 21, 2023
Hi all.
25.0.9 was just released and the fix is still not there - I can't seem to tag the backend team on GitHub, but if there's anything I can do to help here just let me know. I have fixed it manually for me, but I imagine other users are suffering from this without a way to fix it.
Thanks!
pjft commentedon Sep 15, 2023
Hi all. Apologies for the repeated message, but 25.0.11 was just released and I still had to manually patch this. Is there anything I need to do to get this backported to 25?