The current mode used for parallelism if multiple GPUs/TPU cores are available. One of: - `ParallelMode.NOT_PARALLEL`: no parallelism (CPU or one GPU). - `ParallelMode.NOT_DISTRIBUTED`: several GPUs in one single process (uses `torch.nn.DataParallel`). - `ParallelMo
(self)
| 1906 | |
| 1907 | @property |
| 1908 | def parallel_mode(self): |
| 1909 | """ |
| 1910 | The current mode used for parallelism if multiple GPUs/TPU cores are available. One of: |
| 1911 | |
| 1912 | - `ParallelMode.NOT_PARALLEL`: no parallelism (CPU or one GPU). |
| 1913 | - `ParallelMode.NOT_DISTRIBUTED`: several GPUs in one single process (uses `torch.nn.DataParallel`). |
| 1914 | - `ParallelMode.DISTRIBUTED`: several GPUs, each having its own process (uses |
| 1915 | `torch.nn.DistributedDataParallel`). |
| 1916 | - `ParallelMode.TPU`: several TPU cores. |
| 1917 | """ |
| 1918 | requires_backends(self, ["torch"]) |
| 1919 | if is_torch_xla_available(): |
| 1920 | return ParallelMode.TPU |
| 1921 | elif is_sagemaker_mp_enabled(): |
| 1922 | return ParallelMode.SAGEMAKER_MODEL_PARALLEL |
| 1923 | elif is_sagemaker_dp_enabled(): |
| 1924 | return ParallelMode.SAGEMAKER_DATA_PARALLEL |
| 1925 | elif self.distributed_state is not None and self.distributed_state.distributed_type != DistributedType.NO: |
| 1926 | return ParallelMode.DISTRIBUTED |
| 1927 | elif self.n_gpu > 1: |
| 1928 | return ParallelMode.NOT_DISTRIBUTED |
| 1929 | else: |
| 1930 | return ParallelMode.NOT_PARALLEL |
| 1931 | |
| 1932 | @property |
| 1933 | def world_size(self): |
nothing calls this directly
no test coverage detected