File Transfer

From open-smil
Jump to: navigation, search

Jetty Server

SMIL and media files can be transferred to SMIL player by using Jetty. All uploaded files will be stored in a protected storage area on SMIL player and will not be read by other applications.

Upload media files

Get access token

  • Firstly, get access token by RestFul api. All QJetty request from remote side need a auth token:
POST /v1/oauth2/token
  • Example function:
function getToken(ip, password) {
    var param = {
              "grant_type": "password",
              "username": "admin",
              "password": password 
            };
    $.ajax({
        url: "http://" + ip +":8080/v1/oauth2/token",
        method: "POST",
        contentType:"application/json",
        dataType: "text",
        data: JSON.stringify(param)
        })
        .done(successRequestToken)
        .fail(failRequestToken)
}

function successRequestToken(response) {
    var data = JSON.parse(response);
    console.log("token:" + data["access_token"]);  //get access token
}
  • Response JSON String:
{"access_token":"1fs39itt5620oel9hnkp73ukqt","refresh_token":"smjpn0f4iqcnvpbbgjaln8j1en","token_type":"Bearer"}

Make a file header for all media files

The file header of all media files will be updated before the file is being uploaded. If a file header is updated successfully, tht means the file is uploaded to the server and its header is updated. If a header is failed to update, it means file is not uploaded to the server and this file will be uploaded again to the Jetty server.

  • Naming of all media files shall follow format as below:

md5_fileLength.extension ex. "001e86da32880a4772f2146e818d6051_1321.mp3"

  • Folder of all media files is unique. All media files shall put in the "media" folder relative to "index.smil"
  • Following information is needed for make file header:

file_path: relative file path in SMIL file. ex. "media/001e86da32880a4772f2146e818d6051_1321.mp3"

file_size: bytes of file. ex. "1321"

md5: file's md5 value. ex. "001e86da32880a4772f2146e818d6051"

content_type: file's content type. ex. "image/png"

Query content type:

file --mime-type image.png

image.png: image/png

Query file md5:

echo $(md5sum image.png)

001e86da32880a4772f2146e818d6051 image.png

POST /v1/upload/make_file_header
function makefileheader() {
    var fileheaders = [];
    for(var i = 0; i < upload_files.length; i++) {
        console.log("element name:" + upload_files[i].element_name);
        console.log("file_name:" + upload_files[i].file_name);
        console.log("folder:" + upload_files[i].folder);
        console.log("md5:" + upload_files[i].md5);
        console.log("content_type:" + upload_files[i].content_type);
        console.log("size:" + upload_files[i].file_size);
        fileheaders.push({
                            "file_path":upload_files[i].folder + "/" + upload_files[i].file_name,
                            "file_size":upload_files[i].file_size,
                            "md5":upload_files[i].md5,
                            "content_type":upload_files[i].content_type
                        });
    }

    $.ajax({
      url: "http://" + conf.upload_ip +":8080/v1/upload/make_file_header",
      type: "POST",
      contentType:"application/json",
      dataType: "text",
      data: JSON.stringify(fileheaders),
      beforeSend: function(xhr, settings) { xhr.setRequestHeader('Authorization','Bearer ' + conf.access_token); },
    })
    .done(makefileheaderSuccess)
    .fail(printErrResult);
}

Response JSON Array String:

[{"file_path":"media\/b695ff192f123952d7952c8e430f4c32_6997486.mp3","file_size":6997486,"md5":"b695ff192f123952d7952c8e430f4c32","content_type":"audio\/mp3","file_header":false},
{"file_path":"media\/026e6e3f25b3508c2844f31c6be01778_56757.jpg","file_size":56757,"md5":"026e6e3f25b3508c2844f31c6be01778","content_type":"image\/jpeg","file_header":true},
{"file_path":"media\/ed181a40a137dc1e119f0d69f2262c8d_39056349.mp4","file_size":39056349,"md5":"ed181a40a137dc1e119f0d69f2262c8d","content_type":"video\/mp4","file_header":false}]

If "file_header" element return false, it means make header is failed, the file need to upload to server again.


Upload files for make header failed files

Small File (File size <= 50MB)

For small files that failed to make a header, It will be uploaded to server by multipart/form-data format. The upload request will be slipted, if the size of upload files are bigger than 500MB. For example, if there are four media files to be uploaded,

1. 302MB

2. 200MB

3. 1005MB

4. 100MB


Three upload requests will be generated, first request contains item 1&2, (item 1 + item 2 more than 500 MB) Second request contains item 3, (item 3 more than 500 MB) Last is item 4.


The Upload requests will be sent in a sequence. Second request will be waiting until the first request's response is return. The last request wait for the second request return and so on. Send more than one request to the device at the same time is an invalid behavior.


File upload path:

POST /v1/upload

Upload files by multipart/form-data format:

contentType:"multipart/form-data" 

Input element name attribute format:

md5_fileSize_folder (ex. ed181a40a137dc1e119f0d69f2262c8d_39056349_media)

The forder will be named as "media", cannot be modified.

<form role="form" ng-submit="startUpload()" name="uploadForm" enctype="multipart/form-data">
  <li ng-repeat ="file in files track by $index">
    <label><B>{{files[$index].id}}</B></label> <input type="file" name={{files[$index].element_name}} id={{files[$index].id}} onchange="angular.element(this).scope().onInputChange(this)" ng-model="myvalue" required/> 
    <label style="color:#458F20" >MD5:</label><input 
                       ng-required="true" size="32" ng-model="files[$index].md5" readonly />  
    <label style="color:#652F80" >Folder:</label><input 
                       size="20" ng-model="files[$index].folder" ng-change="folder_change(file, $index)" />                       
                       <br /><br />
    <label style="color:#458F20" >Size:</label><input ng-required="true" size="20" ng-model="files[$index].file_size" readonly />  
    <label style="color:#652F80" >Content-Type:</label><input ng-required="true" size="12" ng-model="files[$index].content_type" readonly />  <br /><br /><br />
  </li>
  <button type="submit" class="btn btn-primary pull-right">Try it!</button>
</form>
function makefileheaderSuccess(str) {
    printResult(str);
    var jarray = JSON.parse(str);
    var oReq = new XMLHttpRequest();
    oReq.open("POST", "http://" + conf.upload_ip +":8080/v1/upload", true);
    oReq.onreadystatechange = function() {
        if (oReq.readyState == XMLHttpRequest.DONE) {
            printResult(oReq.responseText);
            for(var i = 0; i < jarray.length; i++) {
                $("#" + upload_files[i].id).removeAttr("disabled");
            }
        }
    }
    oReq.setRequestHeader("Authorization", 'Bearer ' + conf.access_token);
    for(var i = 0; i < jarray.length; i++) {
        console.log("file name:" + jarray[i].file_path);
        console.log("md5:" + jarray[i].md5);
        console.log("content_type:" + jarray[i].content_type);
        console.log("file_header:" + jarray[i].file_header);
        if(jarray[i].file_header) {
            $("#" + upload_files[i].id).attr("disabled", "disabled");
            console.log("remove element");
        }
    }
    var oData = new FormData(document.forms.namedItem("uploadForm"));
    oReq.send(oData);
}

Response JSON String:

[{"field_name":"b695ff192f123952d7952c8e430f4c32_6997486_media","file_name":"b695ff192f123952d7952c8e430f4c32_6997486.mp3","content_type":"audio\/mp3","file_size":6997486,"result":"success"},
{"field_name":"ed181a40a137dc1e119f0d69f2262c8d_39056349_media","file_name":"ed181a40a137dc1e119f0d69f2262c8d_39056349.mp4","content_type":"video\/mp4","file_size":39056349,"result":"success"}]

For result Code:

result: "server_err"|"success"|"overwrite"|"checkfailed"

"server_err":server unexpected error.

"success":upload successfully.

"overwrite":file exist, overwrite the file.

"checkfailed": check sum and file length failed. Need to resend file.

Large File (File size > 50MB)

For large files, split the file and upload it by chunk is more efficient. For a File "foo.mp4", file size: 130MB. MD5 (first 50MB):495a19fc28bce1384c002029b8fd4bf4 Split the file to 3 parts and upload it by multipart format. Header "upload_info": { "part_info": [ { "part_index":0, "part_offset":0, "part_bytes":52428800, "part_md5":"495a19fc28bce1384c002029b8fd4bf4" }, { "part_index":1, "part_offset":52428800, "part_bytes":52428800, "part_md5":"5ef007b127c3b1bd5768999cc72e4a7d" }, { "part_index":2, "part_offset":104857600, "part_bytes":31457280, "part_md5":"40d473e3e72f02576189f64827b1ca2d" } ], "total_bytes":136314880, "total_parts":3, "file_name":"495a19fc28bce1384c002029b8fd4bf4_136314880" }


Content: field_name:495a19fc28bce1384c002029b8fd4bf4_52428800 field_name:5ef007b127c3b1bd5768999cc72e4a7d_52428800 field_name:05b8825669ae9dee519349e4a9edafca_136314880

Response JSON String:

[{"field_name":"495a19fc28bce1384c002029b8fd4bf4_52428800","file_size":52428800,"result":"success"},
{"field_name":"5ef007b127c3b1bd5768999cc72e4a7d_52428800","file_size":52428800,"result":"success"},
{"field_name":"05b8825669ae9dee519349e4a9edafca_136314880","file_size":136314880,"result":"success"}]

For result Code: result: "server_err"|"success"|"overwrite"|"checkfailed" "server_err":server unexpected error. "success":upload successfully. "overwrite":chunk file exist, overwrite the file. "checkfailed": check sum and file length failed. Need to resend chunk file.


Check chunk file is exist or not For chunk file, you can check it exist or not before upload. If the file is uploaded in before, you can skip the chunk. For check chunk file.

Get /v1/upload/chunk/{MD5_LENGTH_TOTALPART}/{CHUNKMD5_CHUNKLENGTH_PARTINDEX}

MD5_LENGTH_TOTALPART: the final file. MD5:Upload file's MD5 of first 50MB. LENGTH:Upload file's file size TOTALPART:Total parts to be split. CHUNKMD5: chunk md5 CHUNKLENGTH: chunk length PARTINDEX: the part number

Example:

Get /v1/upload/chunk/495a19fc28bce1384c002029b8fd4bf4_136314880_3/5ef007b127c3b1bd5768999cc72e4a7d_52428800_1

Result:

{"file_name":5ef007b127c3b1bd5768999cc72e4a7d__52428800_1, "file_length":5248800}

Merge chunk

After all chunks are uploaded, you can merge it to the complete file by following command

Post /v1/upload/chunk/MD5_LENGTH_TOTALPART/chunk_done

For example:

Post /v1/upload/chunk/05b8825669ae9dee519349e4a9edafca_136314880_3/chunk_done

Result:

{"result":"success"}


result: "server_err"|"success"|"checkfailed"|"miss" "server_err":server unexpected error. "success":upload successfully. "overwrite":file exist, overwrite the file. "checkfailed": check length failed. All length of parts are not equal to file length "miss":Miss some parts.

Failed result:

{"result":"miss" 
 "miss_part":[0,1]
}

Upload SMIL File

Upload the SMIL file (.smil) when the transfer of the media files is completed.

Change Content URI by RestApi

Change device content URI by following command after SMIL file is uploaded:

POST /v1/settings/smil_content_url

Content:

{
    "value": "http://localhost:8080/content/index.smil" 
}

index.smil is a file name of SMIL file that is uploaded. Device content URL will be changed to "http://localhost:8080/content/index.smil" after the command is exectured successfully.

From a USB drive

Media and SMIL can be transferred to the SMIL player from a USB drive and played locally.

Procedure

Step 1: Copy files to a USB drive

Create a “smil” folder under the root directory of a USB drive. Copy media and SMIL files under the “smil” folder.


Step 2: Connect the USB drive with SMIL player

Contents will be copied to SMIL player automatically.

HTTP pull mode

Check for new contents

SMIL player will send a HTTP request to CMS every minute. If a different header is returned by Server, SMIL player will download the content from CMS and play it locally.

Process flow

HTTP Pull Mode.PNG