How to Compress JSON file using GZIP Compression
Posted By : Ankit Uniyal | 31-Mar-2018
In this blog, we will cover how we can compress JSON file using GZIP compression and upload on S3 bucket, we will also cover how we can decompress s3 JSON file in Node.js. We are using
Below are the methods for achieving GZIP compression and decompression in Node.js :
1. For GZIP Compression and upload GZIP file on AWS S3 bucket:
var getCompressedJSONFile = function(next, jsonFileContent, results) { console.log("getCompressedJSONFile function started", jsonFileContent); var bufferObject = new Buffer.from(JSON.stringify(jsonFileContent)); zlib.gzip(bufferObject, function(err, zippedData) { if (err) { console.log("error in gzip compression using zlib module", err); next(err); } else { console.log("zippedData", zippedData); next(null, zippedData); } }) };
Upload compressed GZIP file on S3 bucket :
var uploadEntitlementDataOnS3 = function(next, compressedFileContent, results) { console.log("uploadEntitlementDataOnS3 function started", compressedFileContent); userEmailId = userEmailId.replace(/[\. ,@:|&-]+/g, "-"); console.log("userEmailId is", userEmailId); var filePath = configurationHolder.config.s3UploadFilePath; var s3 = new AWS.S3(); var params = { Bucket: configurationHolder.config.bucketName, Key: filePath, Body: compressedFileContent, ContentType: "application/json", ContentEncoding: 'gzip' } s3.putObject(params, function(err, data) { if (err) { console.log(err, err.stack); next(err); } else { next(null, filePath); } }); };
2. Decompress GZIP file using
var deCompressedJSONFile = function(next, compressedFileContent, results) { console.log("deCompressedJSONFile function started", compressedFileContent); zlib.unzip(compressedFileContent, function(err, unZippedData) { if (err) { console.log("error in unzip decompress using zlib module", err); next(err); } else { console.log("unZippedData", unZippedData); next(null, unZippedData); } }) }
Thanks
Cookies are important to the proper functioning of a site. To improve your experience, we use cookies to remember log-in details and provide secure log-in, collect statistics to optimize site functionality, and deliver content tailored to your interests. Click Agree and Proceed to accept cookies and go directly to the site or click on View Cookie Settings to see detailed descriptions of the types of cookies and choose whether to accept certain cookies while on the site.
About Author
Ankit Uniyal
Ankit has knowledge in Javascript, NodeJS, AngularJS and MongoDB also have experience in using AWS Services.