My Learning's with OpenVino

I have worked with Openvino, Rasberry Pi for my projects. Sharing some of my perspectives on OpenVino.

Pros

• Support Multiple Frameworks (Pytorch, Caffe, TensorFlow, etc)
• With the inbuilt Inference Engine reduce model runtime with Model Optimization. Competitor for Edge Based Platforms From Nvidia, Google TPU, etc..
• Support both Windows / Linux platforms

To be Improved

• Better documentation. Provide a detailed Sample code to invoke a model. Provide detailed documentation similar to Opencv Examples. Example below to detect pedestrains using Openvino

def Detect_Pedestrians_Adas():
		attr_bin = '/opt/intel/computer_vision_sdk/deployment_tools/intel_models/pedestrian-detection-adas-0002/FP32/pedestrian-detection-adas-0002.bin'
		attr_xml = '/opt/intel/computer_vision_sdk/deployment_tools/intel_models/pedestrian-detection-adas-0002/FP32/pedestrian-detection-adas-0002.xml'
		ped_net = cv2.dnn.readNet(attr_bin, attr_xml)
		ped_net.setPreferableTarget(cv2.dnn.DNN_TARGET_CPU)
			
		print("Models loaded")
		files = os.listdir(DATA_DIR)
		#get all files in directory
		#loop through for all files
		i = 0
		for file in files:
			filepath = DATA_DIR + '//'+ file
			print(filepath)
			frame = cv2.imread(filepath)
			#https://docs.openvinotoolkit.org/latest/_models_intel_pedestrian_detection_adas_0002_description_pedestrian_detection_adas_0002.html
			blob = cv2.dnn.blobFromImage(frame,size=(672,384),ddepth=cv2.CV_8U)
			ped_net.setInput(blob)
			out = ped_net.forward()
			predictions = []
			for detection in out.reshape(-1,7):
				image_id,label,conf,x_min,y_min,x_max,y_max = detection
				if conf > 0.5:
					predictions.append(detection)
			#print(predictions)
			print(len(predictions))
			for detection in predictions:
				confidence = float(detection[2])
				xmin = int(detection[3]*frame.shape[1])
				ymin = int(detection[4]*frame.shape[0])
				xmax = int(detection[5]*frame.shape[1])
				ymax = int(detection[6]*frame.shape[0])
				print(xmin,ymin,xmax,ymax)
				pedestrian = frame[ymin:ymax,xmin:xmax]
				result_filepath = RESULTS_DIR + '//'+ str(i) + '.jpg'
				#write the output in directory
				cv2.imwrite(result_filepath,pedestrian)
				cv2.rectangle(frame,(xmin,ymin),(xmax,ymax),color=(0,255,0))
				i = i+1
			cv2.imshow("Result",frame)
			cv2.waitKey(0)
			cv2.destroyAllWindows()

• No details on the Dataset/model architecture of Openvino Models shared. Without these details it's very difficult to customize or improve the model
• Porting Errors - I have faced multiple times of error in porting models. I have got resolutions from forums but still, I feel these need to be working fine without issues
• I was not able to run my model inference in Linux which I ported from Windows :). I will reach out to forums for help.

I feel Raspberry pi + Tensorflow Lite a good alternative edge device combination. A lot of improvements have been done and TensorFlow now runs in Microcontroller too. I was able to do a lot of out of box models in the Raspberry pi version 4 device. 4GB RAM and 32GB disc were good enough to run TensorFlow models. 

Hoping to experiment more. I am more inclined towards RasberryPI + Tensorflow Lite combination because you have access to model architecture, datasets used, broader community and lot of traction in this space. Intel has both commercial and opensource version of OpenVino. Like a blackbox, it will be difficult to take a model and implement it. I would rather prefer to evaluate architecture, datasets/customize and implement it. These are my reasons to prefer PI + Tensorflow over Openvino. Tensorflow Lite vs Openvino benchmark would be a good attempt to quantify the numbers. I am hoping to do the same in the near future.

Happy Learning!!!