How to upload from BUFFER?

@russian-dima all clear, thanks! Editted the solution. Will e-mail you on overcoming of the field limitations though.

@vervoortyves You got a message! :wink:

Like most I find myself reading this post because I have a use case where
I need to specify the path name of the file I upload.

We initially look at the upload button, but its function for https://www.wix.com/velo/reference/$w/uploadbutton/startupload
won’t allow this, it places everything in root ‘Vistor Uploads’ folder.

This article looks like the right solution but leave some big learning curve challenges between using the uploadbutton and this backend code.
a) how does one create and lunch a media sector/file picker to place in the buffer? (now we no longer use the upload button)
b) what was the actual answer to how to encode that buffer correctly. Not here, instead left to be “figured out” in what quite a messy post.
c) its backend code, we need to do something about using this and we cannot in preview? Or we need to have separate front end code somewhere?

any help on the questions above is appreciated.

Anybody said that this one is a simple task to solve within a few hours.
Please read all the posts and take carefully onto every little hint, which you can find. You will find everything you need in the post, to solve your issue.

Why this post is messy? Perhaps because it is the first and only one which handle this issue. I am almost sure you won’t find any other post which handles this thematic in such an detailed maner.

Your steps would be the following to solve this issue.

  1. Creating an HTML-Component-File-Selector, which gather all needed informations including the base-64-code.
  2. Sending all gathered data to your site.
  3. Using the provided code from the Wix-Api on back-end (send DATA to backend).
  4. Handle the data on backend.

@chris-derrell

Hey!
I am trying to follow your thoughts and implement image upload to MediaManager via getUploadUrl() . The standard method (via upload () on backend ) does not suit me because of the file size limitations, as you wrote above.

So I have:

  1. HTML component , that read the selected image and convert it to base64;
  2. getUploadUrl.jsw on backend, which get an upload link for me;
  3. And that part
import * as request from 'request-promise-node';

async function uploadImageViaUploadUrl(uploadUrl, uploadToken, contentStream, fileName, contentType) {
  const body = {
    upload_token: uploadToken,
    file: {
      value: contentStream, // a Node.js Buffer with the file content
      options: {
        filename: fileName,
        contentType: contentType
      }
    }
  };

  const response = await request.post({url: uploadUrl, formData: body, json: true});
  return `wix:image://v1/${response[0].file_name}/${response[0].original_file_name}#originWidth=${response[0].width}&originHeight=${response[0].height}`;
} 


And my goal is for it to work like this:
The user selects an image, the HTML component requests an upload link, the getUploadUrl.jsw module generates an upload link and sends it back to the HTML component so that the component sends a request to upload the file by this link.

BUT, I don’t know how and where I should use the part with uploadImageViaUploadUrl
?
Also, do I have to fetch smth somewhere to get uploadImageViaUploadUrl work?
How should I do that?

Thanks for any help!

@ab4583325
I thought, this was solved already?

@ab4583325 Nice work, so far. A couple of questions to better give you guidance.

Is the user selecting the image within the HTML Component? If not, having them select the image that would be the best place, so that you will not have to pass the data between your Wix page and the HTML Component via .postMessage(). That’s important because Post Message also has a limit on the size of the string you can pass.

If you’re already doing that, you can try the following.

1. Create An Upload Form in the HTML and Send Upload URL to HTML Component

You can send the generated Upload URL over to the HTML Component using https://www.wix.com/velo/reference/$w/htmlcomponent/postmessage

Once you’ve sent it, you can then take the URL inside the component and set it to the action of the form.

I’ve written some code for the HTML Component to guide you. You’ll have to set the uploadURL to your own in the component. You’ll need to https://stackblitz.com/edit/web-platform-tmegez?file=index.html

Why a form Chris?
What about the Data URL?

The API is looking out for a FormData formatted body in the post data, which you can do easily by using an HTML Form. If you definitely don’t want to create a form just to upload, you can also create formData using Javascript, within the HTML Component Using FormData Objects - Web APIs | MDN

@russian-dima

Hello!
I solve this via your solution
(like this

  1. Creating an HTML-Component-File-Selector, which gather all needed informations including the base-64-code.
  2. Sending all gathered data to your site.
  3. Using the provided code from the Wix-Api on back-end (send DATA to backend).
  4. Handle the data on backend.)

But this way I can send only 2-3MB image (for larger images I get Uncaught in promise ), and for larger images as I understand i need to use getUploadUrl to create link to upload by performing a POST operation to that URL.
BUT apparently

import * as request from 'request-promise';

const response =await request.post({url: uploadUrl, formData: body, json:true});

this construction doesn’t work anymore.
request-promise was deprecated
And now I try to use node-fetch npm

import fetch from 'node-fetch';

export async function uploadImageViaUploadUrl(uploadUrl, uploadToken, contentStream, fileName, contentType) {
  let buf = Buffer.from(contentStream, "base64");
  const body = {
    upload_token: uploadToken,
    file: {
      value: buf, // a Node.js Buffer with the file content
      options: {
        filename: fileName,
        contentType: contentType
      }
    }
  };
  await fetch( uploadUrl, {
  method: 'post',
  body: JSON.stringify(body),
  headers: {'Content-Type': contentType}
  })
  .then( (httpResponse) => {
    if (httpResponse.ok) {
      return httpResponse.json();
    } else {
      return Promise.reject("Fetch did not succeed");
    }
  } )
  .then( (json) => console.log(json.someKey) )
  .catch(err => console.log(err));

But no luck here, all I get is Fetch did not succeed.
I think the problem is in node-fetch or my fetch creation.
Stuck here.

@chris-derrell
Thank you very much!
So

Is the user selecting the image within the HTML Component?
Yes, that’s done.

1. Create An Upload Form in the HTML and Send Upload URL to HTML Component
That’s also done now.

I’ve written some code for the HTML Component to guide you.
Thank you so much I will try it right now!:eyes:

@ab4583325 how’s it going?

@ab4583325 Will follow your progress…

@chris-derrell
@russian-dima

Ok, I think that there is very little left to win.
Now my HTML-component looking like that:

<html>
<head>
    <script type="text/javascript">
//PART 01
var openFile = function(event) {
    var msg = {fileName:"", fileType:"", content:"", fileSize:"", accepted:""};
    var input = event.target; 
    var reader = new FileReader();
    var fileName = input.files[0].name;   
    var fileType = input.files[0].type; 
    var fileSize = input.files[0].size;
    var accepted = input.accept;

    reader.readAsDataURL(input.files[0]);

    reader.onloadend = function sendmsg(){
      var buffer = reader.result;   
      msg.fileName = fileName; 
      msg.content = buffer; 
      msg.fileType = fileType;
      msg.fileSize = fileSize;
      msg.accepted = accepted;
      window.parent.postMessage(msg,"*");
      console.log("MSG: ", msg)
    };
//PART 02    
    window.onmessage = (event) => {
    console.log(event.data);
    var url = event.data;
    var formData = new FormData();
    formData.append("upload_token", "");
    var file = {
      value: msg.content,
      options: {
        filename: msg.fileName,
        contentType: msg.fileType
      }
    }
    formData.append("file", file);
    let request = new XMLHttpRequest();
    request.open("POST", url);
    request.send(formData);
  };
  };

  </script>
  </head>

<body>
<label class="custom-file-upload" >
   <input type="file" accept='image/*' onchange='openFile(event)' />
  </label>
</body>
</html>

Part 01 - working on reading the image in base64 and sending message to the front.
Part 02 - working on receiving the message from the front with the url to upload to, and preparing data to Post request.

If I run this code now I get
POST https://upload.wixmp.com/upload/eyJhbGciOi… 500 ERROR

I think the problem is in

request.send(formData);

I can not correctly prepare formData for sending.
I would be grateful for any hints!

Ok, I think now it’s better, but how to import Buffer npm node here?
Or create buffer somehow else?

<html>
<head>
    <script type="module">
var openFile = function(event) {
    var msg = {fileName:"", fileType:"", content:"", fileSize:"", accepted:""};
    var input = event.target; 
    var reader = new FileReader();
    var fileName = input.files[0].name;   
    var fileType = input.files[0].type; 
    var fileSize = input.files[0].size;
    var accepted = input.accept;

    reader.readAsDataURL(input.files[0]);

    reader.onloadend = function sendmsg(){
      var buffer = reader.result;   
      msg.fileName = fileName; 
      msg.content = buffer; 
      msg.fileType = fileType;
      msg.fileSize = fileSize;
      msg.accepted = accepted;
      window.parent.postMessage(msg,"*");
    };
    window.onmessage = (event) => {
    var url = event.data;
    var img = msg.content; 
    image = img.split(`data:${msg.fileType};base64,`)[1];
    let buff = Buffer.from(image, "base64");
    var formData = {
    upload_token: "",
    file: {
      value: buff, 
      options: {
        filename: msg.fileName,
        contentType: msg.fileType
      }
    }
  };
    let request = new XMLHttpRequest();
    request.open("POST", url);
    request.send(formData);
  };
  };

  </script>
  </head>

<body>
<label class="custom-file-upload" >
   <input type="file" accept='image/*' onchange='openFile(event)' />
  </label>
</body>
</html>

So on the bright side for you, you don’t need to use the NPM buffer library, because the FileReader api has one of its own. Instead of reader.readAsDataURL, you should use reader.readAsArrayBuffer.

That will give you the buffer directly to use in the request, no conversion necessary

@chris-derrell

Thanks a lot!
Now I’ve got correct Buffer, but after POST still getting 500 ERROR.

<html>
<head>
    <script type="text/javascript">
var openFile = function(event) {
    var msg = {fileName:"", fileType:"", content:"", fileSize:"", accepted:""};
    var input = event.target; 
    var reader = new FileReader();
    var fileName = input.files[0].name;   
    var fileType = input.files[0].type; 
    var fileSize = input.files[0].size;
    var accepted = input.accept;

    reader.readAsArrayBuffer(input.files[0]);

    reader.onloadend = function sendmsg(){
      var buffer = reader.result;   
      msg.fileName = fileName; 
      msg.content = buffer; 
      msg.fileType = fileType;
      msg.fileSize = fileSize;
      msg.accepted = accepted;
      window.parent.postMessage(msg,"*");
      console.log("MSG: ", msg)
    };
    window.onmessage = (event) => {
    console.log(event.data);
    var url = event.data;
    var img = msg.content; 
    var formData = {
    upload_token: "",
    file: {
      value: msg.content, 
      options: {
        filename: msg.fileName,
        contentType: msg.fileType
      }
    }
  };
    console.log(url);
    console.log(formData);
    let request = new XMLHttpRequest();
    request.open("POST", url);
    request.send(formData);
  };
  };

  </script>
  </head>

<body>
<label class="custom-file-upload" >
   <input type="file" accept='image/*' onchange='openFile(event)' />
  </label>
</body>
</html>

Do I need an UploadToken? I doubt.
So my formData still incorrect?

In console formData look like that

file:
    options: {filename: "05f3ef4974016ea1efdc5862bd14b5c6.jpg", contentType:         "image/jpeg"}
    value: ArrayBuffer(84008)
    [[Prototype]]: Object
upload_token: ""
    [[Prototype]]: Object

@chris-derrell

You posted above that token not necessary to upload a file by POST. Could you comment on this in more detail? It seems to me that getUploadUrl SHOULD give a token and the problem here is that we are knocking on URL without a proper token.
What do you think? Maybe it’s a getUploadUrl bug?

Thanks!

Started a new discussion about this
https://www.wix.com/velo/forum/coding-with-velo/getuploadurl-doesn-t-return-a-token

I was going to suggest the same. Closing this thread