Category: Misc

Hi everyone. I’m deploying a resnet based 928×928 UNet on an android device. Performance is suboptimal even with GPU. Currently I’m only optimizing the models using the tf.lite.Optimize.DEFAULT flag. I was wondering if any of you have had experience with more intricate optimization techniques aimed specifically at latency and not neccesarily size reduction.

I see multiple options on the internet to optimize inference, and i don’t know which would be the best fit for me. My goal is to maximize throughput on GPU, and preferably reduce GPU memory usage.

I have a reinforcement learning project, where i have multiple cpu processes generating input data in batches and sending them over to a single GPU for inference. Each process loads the same resnet model with two different weight configurations at a time. The weights used get updated about every 30 minutes and get distributed between the processes. I use Python and Tensorflow 2.7 on Windows(don’t judge) and the only optimization is use right now is the built-in XLA optimizations. My GPU does not support FP-16.

I have seen TensorRT being suggested to optimize inference, i have also seen TensorflowLite, Intel has an optimization tool too, and then there is Tensorflow Serve. What option do you think would fit my needs best?

Hi all,

I’ve got a Lenovo Legion laptop with an onboard GeForce GTX 1660 GPU. Here’s some setup details:

– Ubuntu 21.10

– Python 3.9.7

– using pip (not Conda)

– Tensorflow 2.7.0 (from Python: “tf.__version__” returns 2.7.0)

– TF doesn’t yet recognize GPU existence: “tf.config.list_physical_devices(‘GPU’) returns []

– I think I have CUDA installed: (cat /proc/driver/nvidia/version):

NVRM version: NVIDIA UNIX x86_64 Kernel Module 495.29.05 Thu Sep 30 16:00:29 UTC 2021

GCC version: gcc version 11.2.0 (Ubuntu 11.2.0-7ubuntu2)

I’m doing a TensorFlow tutorial (with PyTorch to come) & have reached a point where I need the GPU. How can I get TF to recognize it?

Before you ask: yes, I *could* download a Docker container or use Colab. I’m going this route because it seems dumb to have a GPU at my fingertips and not use it.

Thanks all & HNY…

I’ve been trying to install tensorflow on vscode for 2 hours now and no matter what I do It wont do it. If someone knows how to do so can they dm me and help.