api documentation updates - devel branch

@raix

While implementing v2 in an app, I'm coming across some irregularities and parts that I seem to be missing in the documentation or needs clarification, so I thought starting a thread to gather them here is useful, feel free to chip in, to help new adopters.

@raix @aldeed Can we keep this issue/thread open for now, to have a single go to place for api docs updates for the devel branch? If so, then I'll simply add issues here one by one, as I stumble upon them.

Missing

After having uploaded files serverside, eg. an image file for use as a graphics element in an app, a natural/intuitive/naive next step seems to be to show it (the single image) in a template. Either via a direct like as if it was uploaded to /public, or via a helper, eg. Template.[template].image = function () { return [fsCollectionName].findOne({_id: id}).url(); }; Maybe there is a better implementation already? What is the way to do this?

Irregularities

cfs-handlebars: is it still recommended or is it deprecated/obsolete due to integrated UI helpers in the new v2 API? (the current doc recommends it)
Insert One File From a Remote URL mentions FS.File.fromUrl, however this no longer works.

@casperanderseneu I thinks its a good idea - The issue will prop. grow long (we had a similar thread called "rewrite" - a google group could perhaps solve it better but what ever works)

So the id in the helper comes from some other document? I would say that the preferred way would be:

  {{#with foo}}
    My name is {{name}} and my image <img src="{{image.url store='thumbnail'}}">
  {{/with}}

Near full example (not tested but should show the intended use)

  // We depend on gm - can this be done from app level yet?
  var gm = Npm.require('gm');

  // We have a regular meteor collection
  var fooList = new Meteor.Collection('fooList');

  // Our images
  var images = new FS.Collection('images', { stores: [
    new FS.FileSystem('thumbnail', {
      path: '/path/to/images',
      transform: function(fileObj /* or readStream, writeStream? */) {
        // This api is WIP not implemented yet - 
        gm(fileObj.createReadStream())
        .resize(10)
        .stream()
        .pipe(fileObj.createWriteStream('thumbnail'));
      }
    })
  ]});

 // Add some data - this would normally be done via events

// Insert + upload a file
 var fileObj = images.insert(file);

 // Insert the mounted image into the normal collection
 fooList.insert({ name: 'bar', image: fileObj });

  // Template helper
  Template.hello.foo = function() {
    return fooList.findOne();
  };

cfs-handlebars - is deprecating should be obsolete at some point - The current api is tightly integrated with the way Blaze works - We have added a package for dropped events - ui-dropped-event there is a very short code example on the package read me (only thing there is really)
Correct fromUrl no longer works - we use the attachData api now

  var fileObj = new FS.File({ name: 'myfile.ext' });
  fileObj.attachData(/* file/blob/Buffer/readStream/url */);
  images.insert(fileObj);

  // I think we will also support this directly or already does?
  var fileObj = images.insert(/* file/url */);
  var fileObj = images.insert('filename.ext', /* file/blob/Buffer/readStream/url */);

One point we need to add / stress in the docs - don't write your own Meteor.methods for insert/remove etc. its a common mistake to think thats any safer than letting the client-side api do the works. Its more secure to actually use the client-side api since we do check allow/deny rules on the server - and its basically Meteor.methods behind the scenes any way.

We are also talking about deprecating cfs-graphicsmagic since the streaming api in the transform function is basically pure gm api.

It would be possible to create transform helper functions eg. in separate packages:

// Our simple transformation function for resizing images
FS.Transform.resizeImage = function(size) {
  // Maybe it makes sense to serve the read and write stream
  return function(fileObj, readStream, writeStream) {
        // This api is WIP not implemented yet - 
        gm(readStream)
        .resize(size)
        .stream()
        .pipe(writeStream);
      }
};

Would simplify api:

  // Our images
  var images = new FS.Collection('images', { stores: [
    new FS.FileSystem('thumbnail', {
      path: '/path/to/images',
      transform: FS.Transform.resizeImage(10)
    })
  ]});

@casperanderseneu, just to add one thing, we would accept PR to update the README. At the moment, the API has been changing almost daily so we have not yet bothered to update everything, but it will smooth out within 2 weeks. Often it seems that someone trying to understand from outside the development group is the best qualified to write documentation because they know what questions people will have.

But that said, we plan to release around the end of the month, and the docs will be updated at that point. We'll use this issue as a point of reference to see what we need to explain better.

@aldeed

I would say ok for meteor-collectionFS or meteor-cfs?

Docs would be ok too - agree that QA/tests/docs are tightly connected - I haven't tested https://hackpad.com but it may be useful in documentation case? - since (as I get it) we could discuss while forming a spec/project/task and it got syntax highlighting? thats the nice thing about having the discussion in git issues

Note: we have been discussing strategy for version numbers etc. target is 1.0.0 - this way we can follow the major version of Meteor (same goes for packages) - we have been using v1/v2 we should really have been saying "old" / "new" api I guess. My bad - so v1.0.0 release.

@aldeed could you confirm? I'm unsure if we actually agreed on this in reality or just in my head, I may have continued our discussion on my own :)

@raix

Truth is I really don't like google groups, but whatever works best for the most people is fine with me. IMO, github issues (which we can tag with discussion or question tag) serve the same purpose and give better code markdown, etc.

Yes, @raix, that sounds like what we decided about versions. :)

So you vote for keeping it simple? I'm fine with all - we could use the meteor talk for community discussions?

MDG uses google groups for feature requests - that pattern doesn't work well, its a black hole for new ideas - so we should avoid gg when discussing something we want to keep a track of.

We can tag in gg but it doesn't have syntax highlighting - why I like github - but discussion threads can become very long - why I'm thinking about hack pad for arc discussions etc. but haven't had the time to check it out.

For the section "Securing Based on User Information" , has that code changed? I'm trying to attach Meteor userIDs to images.

Also, is there a way to limit how many files a user can upload?

@lleonard188

@lleonard188, that example should still work. For limiting uploads, security for insert is done through allow/deny as if it's a normal collection, so you can run a search in a deny function to see how many files already belong to that user and return true if they already have the maximum. Also, for the example in the readme, if you use that code on the client, you would want to verify the owner ID is the current userId in a deny function as well, since client could set it to any user.

You client code could also do a redundant check for max uploads if all the user's files are published to the client, which would save you the server trip and processing.

@aldeed

@aldeed I'm still a bit confused. From what I've read the code below is what I've come up with for an upload-image-and-display-it app, but I don't know how I would find the images belonging to a particular user. Thanks in advance for any help.

HTML

<template name="imageUploader">
    <h2>Picture</h2>
    <p>
    {{#each images}}
            <img src="{{url}}" alt="">
            {{cfsDeleteButton}}
        {{else}}
            No files uploaded.
    {{/each}}
    </p>
    <p>Upload profile pic:</p>
    <p>{{cfsFileInput myImageFiles metadata}}</p>
</template>

client

var imageStore = new FS.Store.S3("images");

var Images = new FS.Collection("images", {
    stores: [imageStore],
    filter: {
        allow: {
            contentTypes: ['image/*']
        }
    }
});

Meteor.subscribe("images");

Template.imageUploader.myImageFiles = function() {
    return Images;
};

Template.imageUploader.images = function () {
    return Images.find();
};

Template.imageUploader.metadata = {
    userId:Meteor.userId()
};

server

var imageStore = new FS.Store.S3("images", {
    region: "us-east-1",
    accessKeyId: "***********", 
    secretAccessKey: "*********", 
    bucket: "*******"
});

var Images = new FS.Collection("images", {
    stores: [imageStore],
    filter: {
        allow: {
            contentTypes: ['image/*']
        }
    }
});

Images.allow({
    insert: function(userId) {
        return !!userId;
    },
    update: function(userId) {
        return !!userId;
    },
    remove: function(userId) {
        return !!userId;
    },
    download: function(userId) {
        return true;
    },
    fetch: []
});

Meteor.publish("images", function() {
    return Images.find();
});

@lleonard188

@lleonard188: image-uploader example gist

I just went through the main package readme and attempted to get all the errors corrected. Still needs work but in general the API examples should be correct as of this moment.

Cool 👍 The api is settling, feels nice - Auto tests should be up and running, cfs-file may need some love, We need to work a bit on the worker etc. I'll try to update milestones etc. To get some overview of status.

Basically what you want to do is download the binary and attach it to the file object so that you can display it from a data URI. So:

Download the data:

Use fileObj.url({store: yourStoreName}) to get the URL. Then use an XMLHttpRequest to get the arraybuffer data from that URL.

Attach the data:

Pass the arraybuffer to attachData. (fileObj.attachData(arrayBuffer))

Get data URI:

fileObj.getDataUri(function (uri) {
  //save URI to a reactive variable or set into img src
});

For those first two steps, I think we can add an attachDataFromStore method on the client to make that simple, or maybe attachData({store: storeName}).

Update:
FS.TempStore now uses the Storage Adapter api and defaults to FS.Store.FileSystem or FS.Store.GridFS depending on installed packages. Filesystem is preferred unless cfs-worker is in the project, then FS.Store.GridFS is preferred due to scalability over multiple instances of Meteor.

Added code documentation FS.TempStore https://github.com/CollectionFS/Meteor-cfs-tempstore

I think we have to rework some of the stuff to take better advantage of the simpler streaming api #239
I havent tested that part of the api, but I see a node fs dependency and thats potentially a problem when deploying.

But yes, the docs seems to be ok, we just need to align the api :)

@raix

Two comments, in the above example:

 fileObj.url({store: yourStoreName})

would be great if examples of accessing data from code, not just templates were provided wherever possible . Took me some time to find this.

Also at the moment:

FS.HTTP.setBaseUrl('/files')

causes ReferenceError: mountUrls is not defined

Wasn't sure if it was doc error, as I saw some comments on the update api docs on this. (but not working for me yet)

Last note, @raix and @aldeed - this is becoming an incredibly useful project - right on par with Collections2 - my kudos for an amazing job. Hope we all have stable projects out for meteor 1.0

@aaronjudd

@aaronjudd, I fixed the setBaseUrl issue now.

@raix

@casperanderseneu, currently I'm getting a "RangeError: Maximum call stack size exceeded" error when doing server-side insert of filepath or URL, like your example. I don't know if this is what you're getting, too. I couldn't figure out the cause at the moment, but maybe @raix will have an idea. Further discussion can move to #246.

The current README uses

var newFile = new FS.File();
newFile.attachData(file);

in a few locations. But that throws an error of "DataMan constructor received data that it doesn't support". Changing it to

var newFile = new FS.File(file);

makes it work.

@calculon0

@calculon0, what type is file? Is it a browser File object? DataMan should support it, especially because if you pass it to the FS.File constructor, it passes it along to attachData for you anyway.

@aldeed

@aldeed I think the issue is the error is being thrown when the constructor is empty. I don't think DataMan allows an empty constructor.

OK, I guess that's correct behavior then, but we could add a more specific error.

An example of how to insert a Buffer would be great. I wrestled with it all afternoon:

request.get({url: result.downloadUrl, encoding: null}, Meteor.bindEnvironment(function(e ,r, body){
    var newFile = new FS.File();
    newFile.attachData(body, {type: 'image/png'}, function(error){
        if(error) throw error;
        newFile.name('mypicture.png');
    });
    Images.insert(newFile);

The options object on attachData() could really use a writeup.

@abuddenb

@abuddenb, if you want to submit a PR with readme updates, we would accept that. There are also lots of examples in the unit tests for the cfs-file package, which could be linked to.

https://github.com/CollectionFS/Meteor-cfs-file/blob/master/tests/file-tests.js#L123

Navigation Menu

Repository navigation

api documentation updates - devel branch #206

13 remaining items

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

api documentation updates - devel branch #206

Description

Activity

raix commented on Mar 19, 2014

raix commented on Mar 19, 2014

aldeed commented on Mar 19, 2014

raix commented on Mar 20, 2014

aldeed commented on Mar 20, 2014

raix commented on Mar 20, 2014

lleonard188 commented on Mar 21, 2014

aldeed commented on Mar 22, 2014

lleonard188 commented on Mar 23, 2014

raix commented on Mar 23, 2014

aldeed commented on Mar 24, 2014

raix commented on Mar 24, 2014

13 remaining items

aldeed commented on Mar 26, 2014

raix commented on Mar 29, 2014

raix commented on Mar 31, 2014

aaronjudd commented on Apr 2, 2014

aldeed commented on Apr 3, 2014

aldeed commented on Apr 3, 2014

calculon0 commented on Apr 7, 2014

aldeed commented on Apr 8, 2014

calculon0 commented on Apr 8, 2014

aldeed commented on Apr 8, 2014

abuddenb commented on Aug 22, 2014

aldeed commented on Aug 22, 2014

Metadata

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

Issue actions