Repository navigation
Persist Content-Type of resources without file extensions #275
Description
(Creating a tracking issue for PRs #159 and #229)
Problem: When writing a resource without a file extension, there is currently no way for the server to remember what its content-type was. So, when it's next requested, the server doesn't know what that resource's content type is, and returns text/turtle by default.
Potential Solutions:
- Add the content type to the resource's
.metafile. Have the server check the.meta(if the resource doesn't have a file extension), discover the content type, and return it with the response. - (Route taken by PR WIP - URI to file system mapping #229) Append a transparent file extension to the resource, only when the resource does not include a file extension. For example, if you PUT a resource named
articlewith an HTML content type, the file will be namedarticle.$.htmlon disk. When reading the resource, ldnode detects the special extension (and filters it out), and returns the resource with the correct content-type.
Activity
sandhawke commentedon Mar 7, 2016
Do we have a spec for how resources live on the filesystem? It seems like something a lot of tools might want to agree on, even though it's not the wire protocol, and not everyone has to use a filesystem, of course. My copy-web-to/from-filesystem tool needs the same thing.
dmitrizagidulin commentedon Mar 7, 2016
@sandhawke - What about solid servers that don't use the filesystem at all, like Meccano?
There shouldn't really be a spec for that, since the only way apps (like the migrator app) should be accessing those files is through the API.
sandhawke commentedon Mar 7, 2016
@dmitrizagidulin I'm just talking about the systems that happen to want to join in, in using the same filesystem conventions. Meccano may want to use it, too, for its export-in-a-zip-file and import-from-a-zip-file functions.
Other elements of this spec would be where metadata and acls go. I'm thinking ...$describedby.ttl and ...$acl.ttl (and this can be extended, where basically the word after the $ is the rel text)
So if I GET /foo and get html, and there's a rel=describedby to /foo,meta.ttl and a rel=acl to /foo,acl then I'll create three files
How's that sound, @timbl ?
nicola commentedon Mar 8, 2016
I agree with @dmitrizagidulin. Although it is nice to have everyone being on the same page on how to represent data on the file system, this is beyond the solid spec - in other words, everyone can implement it the way they want, the import/export that you mention should be done in a ldp way, exactly to avoid this problem, where all of this is abstracted.
sandhawke commentedon Mar 8, 2016
I completely agree it wouldn't be part of solid, per se. Certainly it wouldn't affect normal clients at all. That's why I asked if we "have a spec". (Although I guess if you view solid as a collection of specs my phrasing is ambiguous.)
The "filename conventions" or whatever spec would be both useful and potentially confusing. As you guys suggest, people might think it's part of solid and somehow think they have to follow it. I would think the solid spec could be clear enough about that. Solid could point to it and explain what it's for. I'm sure you'd agree it would actually be useful if gold and ldnode and solid-copy (my current name for the command line copy client) all happily used the same filesystem conventions. At the same time, of course, when folks aren't sharing a filesystem between multiple people or multiple tools, they can do as they like, and even if they are they can use different filename conventions and still be solid.
nicola commentedon Mar 8, 2016
One point @sandhawke,
solid-copybecause it is an "LDP" copy, should not know about how the server store filessandhawke commentedon Mar 8, 2016
Indeed, solid-copy uses solid to talk to the remote server. But it needs some convention for how it is going to store files on the local filesystem. And if that convention is the same as gold and ldnode, then it'll avoid some pain for people running their own servers.
elf-pavlik commentedon Apr 24, 2016
IMO all the RDF Sources should allow content negotiation. How they get stored in filesystem, document database, quads store etc. should not concern clients and stay a choice of internal plumbing used by particular server.
I would recommend that ldnode simply stores all the RDF documents as Normalized RDF.
In my prototype of a server I pass documents through https://github.com/rdf-ext/rdf-normalize before storing them. Later I parse them again with https://github.com/rdf-ext/rdf-parser-n3 and depending on content negotiation response gets serialize with one of https://github.com/rdf-ext/rdf-ext#serializers
elf-pavlik commentedon Jun 14, 2016
For RDF Source, shouldn't the server just store the named graph and don't give any importance to the content type of the payload which provided the g-text ?
Only Non-RDF Source seems to need a way to save information about their content type. Maybe RDF Source which they have desribedby relation with, could include triple with predicate similar to as:mediaType
dmitrizagidulin commentedon Jun 14, 2016
How do you picture the server storing the named graph? For node-solid-server, it stores them as files. And files need extensions, otherwise how is the server supposed to know how to serve them back? (The server does not look inside the file, incidentally).
And yes, this is mainly a problem for non-RDF sources.
elf-pavlik commentedon Jun 14, 2016
For RDF Source server could have configured default media type e.g text/turtle. I assume that we talk about cases where client doesn't include Accept header to content negotiate preferred representation.
I think RDF Source and Non-RDF Source deserve different treatment
zoggy commentedon Mar 20, 2017
@elf-pavlik:
I just discover this issue and I this is the way I chose to store the content type of resources posted or put by clients: the contents of the
content-typeheader of the POST or PUT request is added to the.metaassociated to the new/modified resource. I usedc:formatthough, but without strong conviction about this property rather than another one.15 remaining items
Add file extension to generated paths on POST.
Add file extension to generated paths on POST.
csarven commentedon Sep 20, 2017
Alright, sounds reasonable.
Aside: re HTML+RDFa documents (or dokieli-like thing if you will), we'll see how that works out. Its articles, annotations, notifications do use relative values for some of its subjects/objects. Normally, the base URL will resolve to the URL of the document. So, I think it is possible that .html for instance will leak out. That can be overridden of course when processed, but tools like dokieli can't arbitrarily - and in most cases shouldn't - make the call on what is the intended URL and what's preferable. Needless to say, this is not to suggest that seeing or using
.htmlis an error in any way.I'd like to know what happens with GET
/foo(withfoo.htmlon disk). Does it 301/302, or 200 with optionallyContent-Location? If I understand correctly, GET/foois a 404.. and the POST from earlier probably returnedContent-Location: /foo.html.. so, only GET/foo.htmlis a 200?RubenVerborgh commentedon Sep 20, 2017
Yes, it will currently leak out. But that's not a problem if we treat URLs as opaque identifiers. I.e., the server used to return
foo12345and we did not break any of that apart, now the server will returnfoo12345.htmland we still don't break it apart. To the client, the.htmlshould not have any meaning, it might as well have been `foo12345-xyz' or anything else.Fully agree.
Nothing at the moment, because we still need a good mapping between URIs and the file system (#551).
RubenVerborgh commentedon Jul 19, 2018
Implemented in #643, will be fixed by #662.