Blog

  • In this blog, we will cover how we can compress JSON file using GZIP compression and upload on S3 bucket, we will also cover how we can decompress s3 JSON file in Node.js. We are using zlib npm module for achieving GZIP compression in NodeJS.

    Below are the methods for achieving GZIP compression and decompression in Node.js :

     

    1. For GZIP Compression and upload GZIP file on AWS S3 bucket:

               var getCompressedJSONFile = function(next, jsonFileContent, results) {
                   console.log("getCompressedJSONFile function started", jsonFileContent);
                   var bufferObject = new Buffer.from(JSON.stringify(jsonFileContent));
                   zlib.gzip(bufferObject, function(err, zippedData) {
                       if (err) {
                           console.log("error in gzip compression using zlib module", err);
                           next(err);
                       } else {
                           console.log("zippedData", zippedData);
                           next(null, zippedData);
                       }
                   })
               };   
          

    Upload compressed GZIP file on S3 bucket :

    var uploadEntitlementDataOnS3 = function(next, compressedFileContent, results) {
        console.log("uploadEntitlementDataOnS3 function started", compressedFileContent);
        userEmailId = userEmailId.replace(/[\. ,@:|&-]+/g, "-");
        console.log("userEmailId is", userEmailId);
        var filePath = configurationHolder.config.s3UploadFilePath;
        var s3 = new AWS.S3();
        var params = {
            Bucket: configurationHolder.config.bucketName,
            Key: filePath,
            Body: compressedFileContent,
            ContentType: "application/json",
            ContentEncoding: 'gzip'
        }
        s3.putObject(params, function(err, data) {
            if (err) {
                console.log(err, err.stack);
                next(err);
            } else {
                next(null, filePath);
            }
        });
    };
    

    2. Decompress GZIP file using zlib npm module :

    var deCompressedJSONFile = function(next, compressedFileContent, results) {
        console.log("deCompressedJSONFile function started", compressedFileContent);
        zlib.unzip(compressedFileContent, function(err, unZippedData) {
            if (err) {
                console.log("error in unzip decompress using zlib module", err);
                next(err);
            } else {
                console.log("unZippedData", unZippedData);
                next(null, unZippedData);
            }
        })
    }  
          

     

    Thanks

Tags: nodejs

Mobile Applications

Video Content

Bigdata & NoSQL

SaaS Applications

Miscellaneous

Archives


Alexa Certified Site Stats for www.oodlestechnologies.com