Pass a stream to S3 putObject using AWS SDK for JavaScript v3
On a small side project I've been working on I needed to pass a stream to S3 putObject, however at first I couldn't get this to work.
The documentation suggests that body can accept a stream but I kept getting the following error:
NotImplemented: A header you provided implies functionality that is not implemented
In turns out that putObject does support a stream body, however, the length of the stream must be known. In my case I did not know the length of the stream.
Instead you can useUpload
within @aws-sdk/lib-storage
to get this working.
import { Stream } from 'stream';
import { S3 } from '@aws-sdk/client-s3';
import { Upload } from "@aws-sdk/lib-storage";
async function handler(): Promise<void> {
const passThrough = new Stream.PassThrough();
//code omitted for brevity
someFunction().pipe(passThrough);
const upload = new Upload({
client: new S3({}),
params: {
Bucket: 'bucket-name',
Key: 'key',
Body: passThrough
}
});
upload.on('httpUploadProgress', progress => console.log(progress));
await upload.done();
}