Skip to content

ObjectStore: manifest not present in object created via createLargeObject() #328

Open
Open
ObjectStore: manifest not present in object created via createLargeObject()#328

Description

@olivbd
olivbd
opened on Mar 11, 2021

I followed the sample code samples/objectstore/v1/objects/download.php.

<?php
require 'vendor/autoload.php';

$openstack = new OpenStack\OpenStack([
    'authUrl' => '{authUrl}',
    'region'  => '{region}',
    'user'    => [
        'id'       => '{userId}',
        'password' => '{password}'
    ],
    'scope'   => ['project' => ['id' => '{projectId}']]
]);

/** @var \GuzzleHttp\Stream\Stream $stream */
$stream = $openstack->objectStoreV1()
                    ->getContainer('{containerName}')
                    ->getObject('{objectName}')
                    ->download();

file_put_contents('my_dest_file.bin', $stream->getContents());

It's working for single objects created with createObject() but no with createLargeObject() ! The returned stream is empty...

Please could you provide a sample code to download large files?

Activity

  1. olivbd commented on Mar 11, 2021

    @olivbd
    olivbd
    on Mar 11, 2021
    Author

    My bad! The code for download a single object should be same as for download large object!

    So, the problem concerns only uploading large object. It seems that the manifest isn't present in the object stat. I will investigate this.

  2. olivbd
    changed the title [-]How to download object created via createLargeObject() ?[/-] [+]ObjectStore: manifest not present in object created via createLargeObject()[/+] on Mar 11, 2021

  3. olivbd commented on Mar 12, 2021

    @olivbd
    olivbd
    on Mar 12, 2021
    Author

    I uploaded a 50MiB file with the createLargeObject() method and I confirm there is no manifest header!

    $service = $openstack->objectStoreV1();
$options = [
	'name' => 'my_large_file',
	'stream' => new Stream(fopen('my_large_file', 'r')),
	'segmentSize' => 10000000,
	'segmentContainer' => 'my_container_segments'
];

$service
	->getContainer('my_container')
	->createLargeObject($options)
        ->mergeMetadata(["meta1" => "value1"]);
    swift stat my_container my_large_file
                   Account: AUTH_*********
             Container: my_container
                Object: my_large_file
          Content Type: application/octet-stream
        Content Length: 0
         Last Modified: Thu, 11 Mar 2021 18:41:32 GMT
                  ETag: d41d8cd98f00b204e9800998ecf8427e
            Meta Meta1: value1
         Accept-Ranges: bytes
           X-Timestamp: 1615488091.51685
            X-Trans-Id: tx***-00604b25c3
X-Openstack-Request-Id: tx***-00604b25c3

    If I upload the same file with swift command line utility:

    swift upload -S 10000000 -m "meta1:value1" my_container my_large_file
swift stat my_container my_large_file
                   Account: AUTH_*********
             Container: my_container
                Object: my_large_file
          Content Type: application/octet-stream
        Content Length: 52428800
         Last Modified: Fri, 12 Mar 2021 08:30:17 GMT
                  ETag: "169f9812ae243ad6f224cdfdfb0ca460"
              Manifest: my_container_segments/my_large_file/1615316870.353349/52428800/10000000/
            Meta Mtime: 1615316870.353349
            Meta Meta1: value1
         Accept-Ranges: bytes
           X-Timestamp: 1615537816.67280
            X-Trans-Id: tx***-00604b26ad
X-Openstack-Request-Id: tx***-00604b26ad

    We can see that:

    • Manifest header was missing
    • Content Length was 0

    It explains why I'm unable to download it.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions