Download multiple files from Hubzilla cloud

2019-08-12 18:30:40

emanuel@labonneheure.ch

Is there a way to download multiple files from the #Hubzilla #cloud web UI /cloud (for example all files in a folder, without zipping them)? I know, via webdav it's possible, but some users on my hub find connecting via webdav not intuitive or too complicated
!Hubzilla Support Forum !zotlabs.org | Hubzilla Development

cloud

show all 19 comments

2019-08-12 19:31:50

Hans Sandmeier

hans@hub.netzgemeinde.eu

Good question! - I had the same problem, and found no way to download a folder (like e.g. Dropbox) , so I zipped it on my own and uploaded that too.

2019-08-12 19:38:33

Hans W

hansw@hub.libranet.de

wget -r http://somehub/ .

2019-08-12 19:39:59

Hans W

hansw@hub.libranet.de

or using fuse, that way it can be mounted locally.

2019-08-12 19:47:15

Max Kostikov

kostikov@tiksi.net

@Hans W
As for Windows it can mount Hubzilla CDAV without any problems.

2019-08-12 19:50:26

Hans Sandmeier

hans@hub.netzgemeinde.eu

I think, he wanted some solution for non-technically inclined people. So wget and fuse are no option.

2019-08-12 21:46:15

Mario Vavti

mario@hub.somaton.com

Connect to /dav/ with your favourite webdav client...

2019-08-12 23:39:19

Hans W

hansw@hub.libranet.de

Not, hubzilla at libranet does not seem to work :-)

2019-08-13 09:51:01

Emanuel

emanuel@labonneheure.ch

right, I was talking about a solution for non-technically inclined people. For example you have a complete artist dossier in a Hubzilla cloud folder, with portfolio, video materials, several dozens of photos and so on. You are embedding the files individually in your hubzilla posts, cards, articles and so on. You are presenting the photos with the Hubzilla gallery. And now, you want to provide a convinient download link for the whole folder to a event organizer who needs all these files to work with it for making advertisement, flyers, ...
Or you are having a temporal collaboration project with 5 people. 2 are having a channel on your hub, 3 are not. You need to have a central cloud for all your project material, let's say 30 to 50 files of different types. You create a cloud for that on your hub, since you don't want to use google drive or dropbox. You create a guest access token for the 3 external people. In order to do their work, each project member needs to download a part of the files (let's say about 20 files). The external people access the files only through Web UI (It seems connecting via webdav client doesn't work with the guest access token)

2019-08-13 11:04:43

Daniel

enp4s0@besaid.de

A link can only lead to a single file, so you'd probably need some javascript trickery to active multiple links and get the browser to start downloading them all.
From what I've seen on other platforms, the default seems to be to offer a "download folder as zip" option, which is generated on the fly by the server.
How feasible that is probably depends on the size of the folder, I don't know if zip generation can be streamed or the zip has to be completely generated before the download can start (bad for big folders).
Another option would be to generate a tar (compression optional). The whole definition of a tar is to generate a data stream from one or more files/directories. Though I don't think Windows clients have native support for tar files. Pretty much anything else will probably support it though.

2019-08-13 11:31:00

Emanuel

emanuel@labonneheure.ch

On /webpages/, there's a UI for exporting the webpage elements. There, one can individually check which elements one wants to export, or one toggles all elements (choosing on/off). The export there is offered also as a zip format. I think the same UI could be a solution for the cloud. @Daniel, I see the problem with the size of the files. What would be the size limit, what do you think?

2019-08-13 12:40:40

Daniel

enp4s0@besaid.de

The size limit really depends on the server generating that zip, wether it can be streamed and if not streamable, the size of the temporary space available on the server.
If streaming is possible, there's basically no hard limit as the download can start immediately and no temporary disk space is required. But the runtime of the php process during the stream could be an issue and possibly the memory requirements, as I believe the directory of the zip file is at the end and thus needs to be kept in memory during stream generation.

Also generating a zip file with one million files can take a long time, even if each file is only 1 byte and could easily cause a timeout.
There are a lot of variables here, so it really depends on the system you're running this on and the type of files.
Assuming it's mainly some decently sized files (a couple hundred kilobytes till a couple megabytes) and reasonable numbers of files (~500), I believe that should be fine on most systems.

2019-08-13 23:00:32

El Dorado

mike@z.macgirvin.com

Here's what I can tell you. The software did this in 2005 but that's long before it became a decentralised social project and smartphones didn't yet rule the world.

If you end up with a few videos in the folder (likely in this day and age) you can't use PHP to create the zip (tar, whatever) archive. It will whitescreen due to memory. So you have to use an external shell command to construct the archive. At one time we tried to allow this software to be run on XAMPP (etc.) otherwise I would have done this long ago. That's the primary reason I did not. I think we're solidly *nix only now and a number of things won't work on Microsoft platforms any more and as long as you understand and accept that - using a shell command may be the answer. You should still check for the existence of a zip executable in the path just in case it wasn't included in the distro. Make certain the filelist is generated in PHP and checks for permissions. And then of course you need to buffer stream the resultant file and not try to read it in with file_get_contents(). Since there could be large numbers of these files with different contents (due to different permissions) they absolutely need to be cleaned up after the download finishes and probably after a few hours in case it was aborted. I would probably included a unique hash in each filename to ensure that two people with different permissions downloading the same folder at the same time can't possibly end up with files they aren't allowed to see.

The problem with dav incidentally is that the viewer needs to have a local account. Else they can't authenticate.

For completeness there was once a utility (still in the util folder to my knowledge) which was a cli tool that let's you cd and ls to remote archives (thinking about the dav problem) and cp (download) files. I haven't looked in ages and it probably has bitrot and won't work any more, but one could or should potentially add a download folder command to such a thing. It *might* be easier than the zip file and it *might* exist already. iirc it used the API to do its work.

2019-08-15 20:20:16

meiotexto

meiotexto@redmatrix.us

I'm trying to understand how my login came up in this conversation ... did I do something wrong?

2019-08-15 20:33:47

Emanuel

emanuel@labonneheure.ch

@meiotexto do you refer to this?:

/webpages/

It is displayed individually, according to the observers hub url and the observers webfinger. The syntax for it is [observer.baseurl]/webpages/[observer.name]

2019-08-15 20:41:17

Mario Vavti

mario@hub.somaton.com

@Emanuel you should not wrap the observer tag with the [url] tag. Also you definitely want [observer.webname] instead of [observer.name] in your usecase.

2019-08-15 21:23:43

Emanuel

emanuel@labonneheure.ch

@Mario Vavti Thanks. Good to know. My mistake.

2019-08-16 15:08:19

meiotexto

meiotexto@redmatrix.us

OK @Emanuel I get it!

2019-08-23 03:35:58

Hugo Cerqueira

hrcerq@hubzilla.com.br

Connect to
https://hubzilla.com.br/dav/hrcerq

with your favourite webdav client...

Now I'm getting curious. Why would my profile, specifically, be cited on this topic? Was it picked randomly?

2019-08-23 04:07:03

Giacomino

giavec@zotum.net

Hugo, tenta entender os comentários acima e vc terá a sua resposta ;)