[FIXED] Configure a Video Streaming Flask Server hosting on PythonAnywhere

Issue

Hosting --> PythonAnywhere

Working: The server receives images from a client and updates them to a webpage. Another client accesses the web page and can see the video streaming. The server is not saving images it is just showing them on the webpage as it receives them.

Problem --> The image uploading to the server is smooth but as long as I open the webpage the image upload from the client halts. The server is unable to receive images if someone accesses the webpage

Server Code

from flask import Flask, Response, render_template
from flask import request
import cv2
import numpy as np

app = Flask(__name__)


global current_frame
current_frame = None

def generate_frames():
    while True:
        if current_frame is not None:
            ret, buffer = cv2.imencode('.jpg', current_frame)
            if ret:
                frame_bytes = buffer.tobytes()
                yield (b'--frame\r\n'
                       b'Content-Type: image/jpeg\r\n\r\n' + frame_bytes + b'\r\n')

@app.route('/')
def index():
    return render_template('index.html')

@app.route('/video_feed')
def video_feed():
    return Response(generate_frames(), mimetype='multipart/x-mixed-replace; boundary=frame')

@app.route('/upload_frame', methods=['POST'])
def upload_frame():
    global current_frame
    frame = request.data
    nparr = np.fromstring(frame, np.uint8)
    current_frame = cv2.imdecode(nparr, cv2.IMREAD_COLOR)
    return "Frame received and displayed"

if __name__ == '__main__':
    app.run('0.0.0.0')

Client Code

import cv2
import requests
import numpy as np
import time

server_url = "http://danial880.pythonanywhere.com/upload_frame"


cap = cv2.VideoCapture(0)

while True:
    ret, frame = cap.read()

    if not ret:
        print("Failed to capture frame")
        break

    _, buffer = cv2.imencode('.jpg', frame)
    frame_bytes = buffer.tobytes()

    try:
        # Send the frame to the server
        response = requests.post(server_url, data=frame_bytes, headers={"Content-Type": "image/jpeg"})

        if response.status_code == 200:
            print("Frame uploaded successfully")
        else:
            print(f"Frame upload failed with status code {response.status_code}")
    except requests.exceptions.RequestException as e:
        print(f"Error: {e}")

    #cv2.imshow("Webcam Feed", frame)
    time.sleep(0.1)
    if cv2.waitKey(1) & 0xFF == ord('q'):
        break

cap.release()
cv2.destroyAllWindows()

Solution I tried: making the server asynchronous

from flask import Flask, Response, render_template
from flask import request
import cv2
import numpy as np
import asyncio

app = Flask(__name__)


global current_frame
current_frame = None

async def generate_frames():
    while True:
        if current_frame is not None:
            ret, buffer = cv2.imencode('.jpg', current_frame)
            if ret:
                frame_bytes = buffer.tobytes()
                yield (b'--frame\r\n'
                       b'Content-Type: image/jpeg\r\n\r\n' + frame_bytes + b'\r\n')
        await asyncio.sleep(0.01)  # Add a small delay to reduce CPU load

@app.route('/')
def index():
    return render_template('index.html')

@app.route('/video_feed')
def video_feed():
    return Response(generate_frames(), mimetype='multipart/x-mixed-replace; boundary=frame')

@app.route('/upload_frame', methods=['POST'])
async def upload_frame():
    global current_frame
    frame = request.data
    nparr = np.fromstring(frame, np.uint8)
    current_frame = cv2.imdecode(nparr, cv2.IMREAD_COLOR)
    return "Frame received and displayed"

if __name__ == '__main__':
    app.run()

But after asynchronous server no image is displayed on webpage. The simple server was showing the last frame received.

What changes I can make to the server so that I can have a smooth streaming.

Solution

If Flask is too slow, try, for testing, other solutions for handling concurrent requests like Eventlet (pip install eventlet), or Gevent (pip install gevent), and monkey-patch the standard library to become non-blocking.

For Eventlet (using monkey_patch):

import eventlet
eventlet.monkey_patch()

# rest of your code

if __name__ == '__main__':
    app.run('0.0.0.0')

For Gevent (using atch_all):

from gevent import monkey
monkey.patch_all()

# rest of your code

if __name__ == '__main__':
    app.run('0.0.0.0')

---

Using the monkey patch improves the streaming speed but it is not quite smooth. I have tried the same server code on AWS LightSale. Is there any other way to improve?

If the streaming performance improved with monkey patching but is still not entirely smooth, you could try:

• optimizing the video stream encoding: compress frames to reduce their size before sending them to the server. You could adjust the JPEG compression level in cv2.imencode to find a balance between quality and performance. -limiting the frame rate on both the client's capturing and server's sending side to reduce the workload.
• decreasing the resolution of the video frames being processed and transmitted. Higher resolutions significantly increase CPU and bandwidth usage.
• implementing buffering on the client-side to handle intermittent network delays and to provide a smoother viewing experience.
• implement a producer-consumer queue to handle frame processing. That can help in managing the flow of frames and might smooth out the processing.

Here is an example of how to implement frame rate throttling and resolution reduction:

# On the client side:
cap.set(cv2.CAP_PROP_FPS, 15)  # Limit the frame rate to 15 FPS
ret, frame = cap.read()
frame = cv2.resize(frame, (640, 480))  # Reduce the resolution if necessary

# On the server side:
# Implement a frame rate control in generate_frames function
import time
last_time = time.time()
while True:
    current_time = time.time()
    if current_time - last_time > 1.0 / 15:  # Target frame rate of 15 FPS
        last_time = current_time
        # Rest of the loop logic...

If the server is getting requests from multiple clients, consider implementing load balancing. On AWS, this could be managed with Elastic Load Balancing.

Try also to profile your server's performance to identify bottlenecks. Tools like cProfile for CPU profiling and memory_profiler for memory usage can help.

Keep in mind the network speed and latency could be a factor, especially if different geographical regions are involved. AWS LightSail should provide a better network performance compared to PythonAnywhere, especially if you choose a region closer to your clients.

Experiment with different configurations of web servers and WSGI containers like Gunicorn or uWSGI with Nginx. These servers are more capable of handling concurrent connections and can be fine-tuned for performance.

Consider also using WebSocket for a persistent connection between the client and server, which is more suited for real-time data transfer.

Answered By - VonC