[FIXED] Decode video frame encoded by Python to display in Node based webapp

Issue

I use Python 3 and OpenCV to read video and send the frames to a Node.js based web page via socket.

Let's imagine something like this:

cap = cv2.VideoCapture(video_path)
while cap.isOpened():
    ret, frame = cap.read()
    send(frame)

In the send() function, before actually sending, I pass the frame into a normalize() function to convert it to deliverable format for socket IO. The normalize() looks like below:

def normalize(frame):
    d = frame.flatten()
    s = d.tostring()
    b = b64encode(s)
    return b.decode('ascii')

The data was sent as well. In Node.js app, I receive something as below:

How I can convert this encoded string to normal video, or even image?

Thanks.

Solution

To encode an image, use cv2.imencode(). Then you can write that buffer into a base 64 string, and then display that directly with HTML or whatever you want to do with it.

Here's a basic example. First, the Python:

import cv2
import base64

def encode_img(img):
    """Encodes an image as a png and encodes to base 64 for display."""
    success, encoded_img = cv2.imencode('.png', img)
    if success:
        return base64.b64encode(encoded_img).decode()
    raise ValueError("unable to encode image")

That will give you the encoded image in base 64. The string that this returns can be displayed directly with HTML by using the string as the src of an image if you prepend data:image/png;base64, to it.

So for example...

import numpy as np
img = np.uint8(255 * np.random.rand(50, 50))
encoded_img = encode_img(img)
b64_src = 'data:image/png;base64,'
img_src = b64_src + encoded_img

With this random image my img_src becomes:

'data:image/png;base64,0K4jmTgkWXbvuFdzvFp3kAceWNGRnRdwgKAEdhj8w2Nsw1AhdWtX1UntWIMZRSLmmzRGj0RtU3XxFyhVIb0QPDNp+0d+eI8GAmJru4D9NnFpTiWYGb4KVZP8wkngFrSA1cPXVtw4zGLPPODrPcWFbjwtXfuGH7gJZXLrCb6fbBSXZQ8zTCX7DLwv2mE7C5Qz4Yz3nptzAYuF2kihLfWS0d7eEispjjZM+AEAu2ca9Y+LNd5i8r3IoukdF0UsLR+IJuJcDxo1lR6ABn6B5tebOYryt9N4QLgaEYBMCW2qvlJJCUztE+YoT86sisqELEGyN3NSQdO64vGdBBH6auRZvjSxLaKjXv07v8AY+qYT3w5aq8QZo8o0/BKdNE22H72Hd0avD+CUgfoiXdn6ufzWhyRG33wV2N6H4icitNNca4lFkEUIe+EA+ApghJ8lPqTuZnCPpKt+5VFftkWnUTn9daT8BLVJTJagM9L4FqjGxnMJvOVEIgskQHd/MYqdWGDn8oC5GObZwCpa9o8J3a6jeS2usBtQavZkfOhyxghSscOMDS9XqfsKI1/n9MevvwvMiBgt3REui2ifQQfNnyhGN0NwIejyUGTuSaLPGOocqm8Qfp3UDEFFs5x4TSALD5hGU9BZLyPPUY5bsOsOWnPYorpsbLO+KTKJDOjTBqiZnAEZtvZ9dJKW2wZUOOl5OkdOcMx7VnWLlK+SIi2i67zLatsut8Rr5zQo44YiadwfbbmWddLfoVAzjTsx0bOhrv5JkBMvsDEHaDy3LtR3Dfqf/K7GL8njn23ftCrhvTticX0je60VGlZ6YeJr8a45EC7HYvFvGUZoZZMAuUo+XvZJNeHYHiQTHppdPF/qE7fcISTXL4V6BxIBLPy37vbl2brCIIqLKA2Sn7aumHmR4ohkVs2Gas6s1U8VMANpjldNiOis2GF/mlYMSTcUSB8OekXTpXN4NjYg5QXodq0VtMmw8uLsMGJ8J7tpWqDZrOrNoBS2m/fHSVrsZD7m2PQeiuG6vIwp9cywSQ0SIMd22dpexd1p2jk+wVz2bF6n7g+PTQwa+irCQtLJ/YzQyTnM6qBIDWCIA/qTPDiR35OioBVrEHaK7zph0s2gL+GH/mi6mY/NuoLhiXG0H9/cbWxOcLWgLmoRum55jdwb+jMyOGuk0DP7Vdt8ryySRaQA4WMeYq2Mxxmhxng+OEbGFr3yQxVroZnE75RswCdWlOTJaEs8279WkCzKpOG/h/lt8GoDeU1oEiRqzuVxSk227ifp0Zfjb3WNeSg432g0qiEVsJbZgzACufTrXEyWVU9fObca03pUxMAJiaaDfckWlngVVUsKJz8wNcoSwuAvHXRXwqMxGxTiUC5QuypJ9fkFeLySOOf7R0J41FQr225mpPmadFnxOAxRuNwFSLxCivsz7+n6rQJuFM8ww1t72oMyW6zCArvF2Vaa0CGg9T27TRxk9DV3JWj4TbzSgmmBdCpusIIdGMMIYogUgMq9z2695lRsXdG1TWXoB8lgVEjuwz9rz3Od3pY4oYLnljzuRSPe+qP21QeJvZixEdDf0pADGqPn66IXk+WR0c1vtXc8X7R1tvnqBjjPGtSYX09GAwXcvA+LsBR7lca6iA6oxV3KeK1UCQyB0cBUeb8KBp19IwqbbeEOEezqc7UpLNsrOhDp0xOUngPCIRiCxeNZuC0nmVDFxyrydIc9zkUONbEU7JoZdWdT8yGRfpRS/YBrdN5U3PqC8joeEBqPhQNF97EvsSQoGIL0gn+ti9xmm8Lnxvu5+zIRvaiQoSlNLefPe8/1LxKlQlvU0VOylykb2XOif51p3nEJEgoVeRHG5o354qOhVQe0hvAiqssEiOe9XdX6MgR/V/qX0fM2dvU6CcRbK5fVN+lNkHRrGllbbLtnEyAu8Owe3UlSDGfmXH7K9T1018cdInCkVtx9eOAb+jCEziTfU1A6WWfh44xA5WZcUwA2prj7heWa1xxcBfB03W5R2ISMDzn4dKO22zxQvOr7SQyY7DjEAGNq3c18MVwauzB4IsNAG0AKbJTpLwIxM8bYJ3p9ogwJ8sNxU1Aqa6MB2v0pP3QPReNVR9Lpkziy/hwDj9y7UoOy17WgAhbkxYHo+w42I5tqmEgje/3gGed9lU9vS56OlVWW86s256vTYLfNdhdCmoabMewUaRctGBBLvg/giNS3P8CbDQTcVCTtWoGn60sNanCNDHgIhZ4gJVMu2z9igb6gOmVZ997RgOCfSLEUYBQxvzZLXftQAct/q5Yxw6xqb3BdixgOlnwFRRRCja3b9GkMyR1/IZm19GqVYta4sZPhfPWpv4NyfPNTNoyBtq1R2UIL++Bh4uYeVs+oh8IzL/pqVnpnABAqrWPBAMOEuoqbKOmJDdqFZhc8M2tAbtFOGbMm30Vvd8aQQFkREKfbG6S41PCkzqRmDAF9YqzzL0ZvF24ZOp28eOXdJBhqkSY04FHRk+IV05hjFWclNO99oBsZaILw/QXPal6yz+kkZ+glqcOsDyQ+RndtJwGncRpqrJQGscAnLTpdNL1EgTS5xzzoOTI51QOIJ6+Fde4/z5Micm9NRPTMRN/KNLeLj1CMXMCJaniMj0gBmExGfJ58ZX98fWr0PC42WHYmHjGjkuX25+VdXnQ6c8vgbLB/+VQtgXKJz0t7QljeVfOJp0C9BaBlqfsGNmIn97TZsaDfddbVnST7lf7cOrniBIRQQz+waWNG4TenKhad+EeFy5NeKnFpFCGcOoVxvJunMqkvv1IVSvLJnRQRPD8s0prz2tO3ehfp2a2SGscrQMEpt4GJYwBOYPYICdavWhMHTD7YAZgftKbqAFnZn4Af25N96Fvq/sDhZWzHXHwVDo8EceImp/2c+iIQGV6d6Y0WFEN+APqES7QdiFh+y547bei4jrtbW2tGXtTI/OSn5aenExLX23uW2V1bFkL2WiOf8+LCf0dPLXMdXBysbTAICU3JYDViMchEtFViMzHE2SciZsB3WpxQ9xchpPCpyRQhSOJGu/s/8CGLFDOMNyPRJcOQCZmjqz0kvzp7B5glIJ7OXidKmtDNo0DSWIp+OupU5nglYhalOCgP2zElBG5+hTGDowpht0tHtvLSLeWeRqFQING9KVCHedwO9cdPkKXaOxNvy9AagAqteo86q5vQ98DtfE2C4xIZDjgzyLqCd4XAaoPSfeLC1+OFBIbL80ph9v6n0mz3VBJXAvzTsA1JR36tfKH4k3CnGu6CvVf0wmaTLVCkahF9bwAqdt64NKlafM0vQJbeAVJcFEZ/Dez+B96xpoSX0ph8jjp9sknDdp+vnxitOKDSBw=='

Then in a simple HTML file, use this as the image tag's source:

<html>
<body>
    <img src="data:image/png;base64,0K4jmTgkWXbvuFdzvFp3kAceWNGRnRdwgKAEdhj8w2Nsw1AhdWtX1UntWIMZRSLmmzRGj0RtU3XxFyhVIb0QPDNp+0d+eI8GAmJru4D9NnFpTiWYGb4KVZP8wkngFrSA1cPXVtw4zGLPPODrPcWFbjwtXfuGH7gJZXLrCb6fbBSXZQ8zTCX7DLwv2mE7C5Qz4Yz3nptzAYuF2kihLfWS0d7eEispjjZM+AEAu2ca9Y+LNd5i8r3IoukdF0UsLR+IJuJcDxo1lR6ABn6B5tebOYryt9N4QLgaEYBMCW2qvlJJCUztE+YoT86sisqELEGyN3NSQdO64vGdBBH6auRZvjSxLaKjXv07v8AY+qYT3w5aq8QZo8o0/BKdNE22H72Hd0avD+CUgfoiXdn6ufzWhyRG33wV2N6H4icitNNca4lFkEUIe+EA+ApghJ8lPqTuZnCPpKt+5VFftkWnUTn9daT8BLVJTJagM9L4FqjGxnMJvOVEIgskQHd/MYqdWGDn8oC5GObZwCpa9o8J3a6jeS2usBtQavZkfOhyxghSscOMDS9XqfsKI1/n9MevvwvMiBgt3REui2ifQQfNnyhGN0NwIejyUGTuSaLPGOocqm8Qfp3UDEFFs5x4TSALD5hGU9BZLyPPUY5bsOsOWnPYorpsbLO+KTKJDOjTBqiZnAEZtvZ9dJKW2wZUOOl5OkdOcMx7VnWLlK+SIi2i67zLatsut8Rr5zQo44YiadwfbbmWddLfoVAzjTsx0bOhrv5JkBMvsDEHaDy3LtR3Dfqf/K7GL8njn23ftCrhvTticX0je60VGlZ6YeJr8a45EC7HYvFvGUZoZZMAuUo+XvZJNeHYHiQTHppdPF/qE7fcISTXL4V6BxIBLPy37vbl2brCIIqLKA2Sn7aumHmR4ohkVs2Gas6s1U8VMANpjldNiOis2GF/mlYMSTcUSB8OekXTpXN4NjYg5QXodq0VtMmw8uLsMGJ8J7tpWqDZrOrNoBS2m/fHSVrsZD7m2PQeiuG6vIwp9cywSQ0SIMd22dpexd1p2jk+wVz2bF6n7g+PTQwa+irCQtLJ/YzQyTnM6qBIDWCIA/qTPDiR35OioBVrEHaK7zph0s2gL+GH/mi6mY/NuoLhiXG0H9/cbWxOcLWgLmoRum55jdwb+jMyOGuk0DP7Vdt8ryySRaQA4WMeYq2Mxxmhxng+OEbGFr3yQxVroZnE75RswCdWlOTJaEs8279WkCzKpOG/h/lt8GoDeU1oEiRqzuVxSk227ifp0Zfjb3WNeSg432g0qiEVsJbZgzACufTrXEyWVU9fObca03pUxMAJiaaDfckWlngVVUsKJz8wNcoSwuAvHXRXwqMxGxTiUC5QuypJ9fkFeLySOOf7R0J41FQr225mpPmadFnxOAxRuNwFSLxCivsz7+n6rQJuFM8ww1t72oMyW6zCArvF2Vaa0CGg9T27TRxk9DV3JWj4TbzSgmmBdCpusIIdGMMIYogUgMq9z2695lRsXdG1TWXoB8lgVEjuwz9rz3Od3pY4oYLnljzuRSPe+qP21QeJvZixEdDf0pADGqPn66IXk+WR0c1vtXc8X7R1tvnqBjjPGtSYX09GAwXcvA+LsBR7lca6iA6oxV3KeK1UCQyB0cBUeb8KBp19IwqbbeEOEezqc7UpLNsrOhDp0xOUngPCIRiCxeNZuC0nmVDFxyrydIc9zkUONbEU7JoZdWdT8yGRfpRS/YBrdN5U3PqC8joeEBqPhQNF97EvsSQoGIL0gn+ti9xmm8Lnxvu5+zIRvaiQoSlNLefPe8/1LxKlQlvU0VOylykb2XOif51p3nEJEgoVeRHG5o354qOhVQe0hvAiqssEiOe9XdX6MgR/V/qX0fM2dvU6CcRbK5fVN+lNkHRrGllbbLtnEyAu8Owe3UlSDGfmXH7K9T1018cdInCkVtx9eOAb+jCEziTfU1A6WWfh44xA5WZcUwA2prj7heWa1xxcBfB03W5R2ISMDzn4dKO22zxQvOr7SQyY7DjEAGNq3c18MVwauzB4IsNAG0AKbJTpLwIxM8bYJ3p9ogwJ8sNxU1Aqa6MB2v0pP3QPReNVR9Lpkziy/hwDj9y7UoOy17WgAhbkxYHo+w42I5tqmEgje/3gGed9lU9vS56OlVWW86s256vTYLfNdhdCmoabMewUaRctGBBLvg/giNS3P8CbDQTcVCTtWoGn60sNanCNDHgIhZ4gJVMu2z9igb6gOmVZ997RgOCfSLEUYBQxvzZLXftQAct/q5Yxw6xqb3BdixgOlnwFRRRCja3b9GkMyR1/IZm19GqVYta4sZPhfPWpv4NyfPNTNoyBtq1R2UIL++Bh4uYeVs+oh8IzL/pqVnpnABAqrWPBAMOEuoqbKOmJDdqFZhc8M2tAbtFOGbMm30Vvd8aQQFkREKfbG6S41PCkzqRmDAF9YqzzL0ZvF24ZOp28eOXdJBhqkSY04FHRk+IV05hjFWclNO99oBsZaILw/QXPal6yz+kkZ+glqcOsDyQ+RndtJwGncRpqrJQGscAnLTpdNL1EgTS5xzzoOTI51QOIJ6+Fde4/z5Micm9NRPTMRN/KNLeLj1CMXMCJaniMj0gBmExGfJ58ZX98fWr0PC42WHYmHjGjkuX25+VdXnQ6c8vgbLB/+VQtgXKJz0t7QljeVfOJp0C9BaBlqfsGNmIn97TZsaDfddbVnST7lf7cOrniBIRQQz+waWNG4TenKhad+EeFy5NeKnFpFCGcOoVxvJunMqkvv1IVSvLJnRQRPD8s0prz2tO3ehfp2a2SGscrQMEpt4GJYwBOYPYICdavWhMHTD7YAZgftKbqAFnZn4Af25N96Fvq/sDhZWzHXHwVDo8EceImp/2c+iIQGV6d6Y0WFEN+APqES7QdiFh+y547bei4jrtbW2tGXtTI/OSn5aenExLX23uW2V1bFkL2WiOf8+LCf0dPLXMdXBysbTAICU3JYDViMchEtFViMzHE2SciZsB3WpxQ9xchpPCpyRQhSOJGu/s/8CGLFDOMNyPRJcOQCZmjqz0kvzp7B5glIJ7OXidKmtDNo0DSWIp+OupU5nglYhalOCgP2zElBG5+hTGDowpht0tHtvLSLeWeRqFQING9KVCHedwO9cdPkKXaOxNvy9AagAqteo86q5vQ98DtfE2C4xIZDjgzyLqCd4XAaoPSfeLC1+OFBIbL80ph9v6n0mz3VBJXAvzTsA1JR36tfKH4k3CnGu6CvVf0wmaTLVCkahF9bwAqdt64NKlafM0vQJbeAVJcFEZ/Dez+B96xpoSX0ph8jjp9sknDdp+vnxitOKDSBw==">
</body>
</html>

You should be able to see the image.

You can also save these directly as images on your node server with this format. You can use fs.writeFile() in for e.g. to write this b64 string directly to an image file, by specifying an options object with {encoding: 'base64'}. For example, from this tutorial:

import fs from 'fs';
fs.writeFile('image.png', base64Image, {encoding: 'base64'}, function(err) {
    console.log('File created');
});

Of course if you're using other libraries in node to play with the images, you'll need to use some function from that library to decode the images. Probably not too hard to exchange from b64 to bytes, but you'll need to decode those bytes into an image from whatever libs you're using server-side.

Answered By - alkasm