PyTorch

PyTorch is an optimized tensor library for deep learning using GPUs and CPUs. It based on Torch library for applications such as computer vision and natural language processing, primarily developed by Facebook's AI Research lab (FAIR).

A number of pieces of deep learning software are built on top of PyTorch, including Tesla Autopilot, Uber's Pyro, HuggingFace's Transformers, PyTorch Lightning, and Catalyst.

PyTorch provides two high-level features:

• Tensor computing (like NumPy) with strong acceleration via graphics processing units (GPU)
• Deep neural networks built on a type-based automatic differentiation system

PyTorch defines a class called Tensor (torch.Tensor) to store and operate on homogeneous multidimensional rectangular arrays of numbers. PyTorch Tensors are similar to NumPy Arrays, but can also be operated on a CUDA-capable Nvidia GPU. PyTorch supports various sub-types of Tensors.

PyTorch on Google Cloud

https://cloud.google.com/blog/topics/developers-practitioners/pytorch-google-cloud-blog-series-recap

PyTorch on Vertex AI

https://medium.com/nlplanet/training-a-pytorch-model-on-gcp-vertex-ai-ed20df97ce14

PyTorch vs TensorFlow

TensorFlow is developed by Google Brain and actively used at Google both for research and production needs. Its closed-source predecessor is called DistBelief.

PyTorch is a cousin of lua-based Torch framework which was developed and used at Facebook. However, PyTorch is not a simple set of wrappers to support popular language, it was rewritten and tailored to be fast and feel native.

https://towardsdatascience.com/pytorch-vs-tensorflow-spotting-the-difference-25c75777377b

PyTorch on GCP Vertex AI

https://cloud.google.com/blog/topics/developers-practitioners/pytorch-google-cloud-how-deploy-pytorch-models-vertex-ai

PyTorch/XLA

https://cloud.google.com/blog/topics/developers-practitioners/pytorchxla-performance-debugging-tpu-vm-part-1

Cloud TPU VMs

https://cloud.google.com/blog/products/compute/introducing-cloud-tpu-vms

JAX

https://github.com/google/jax

https://www.assemblyai.com/blog/why-you-should-or-shouldnt-be-using-jax-in-2022/

JAX vs Tensorflow vs PyTorch

https://analyticsindiamag.com/jax-vs-tensorflow-vs-pytorch-a-comparative-analysis/

JAX vs PyTorch: Automatic Differentiation for XGBoost

https://towardsdatascience.com/jax-vs-pytorch-automatic-differentiation-for-xgboost-10222e1404ec

JAX vs Julia

https://kidger.site/thoughts/jax-vs-julia/

PyTorch examples

https://github.com/pytorch/examples

PyTorch Video course

https://www.youtube.com/watch?v=GIsg-ZUy0MY

PyTorch on Kubeflow and Vertex Pipelines

https://cloud.google.com/blog/topics/developers-practitioners/scalable-ml-workflows-using-pytorch-kubeflow-pipelines-and-vertex-pipelines

Scalable ML Workflows using PyTorch on Kubeflow Pipelines and Vertex Pipelines

https://cloud.google.com/blog/topics/developers-practitioners/scalable-ml-workflows-using-pytorch-kubeflow-pipelines-and-vertex-pipelines

How To train and tune PyTorch models on Vertex AI

https://cloud.google.com/blog/topics/developers-practitioners/pytorch-google-cloud-how-train-and-tune-pytorch-models-vertex-ai

How to develop with PyTorch at lightning speed

https://cloud.google.com/blog/products/ai-machine-learning/increase-your-productivity-using-pytorch-lightning