Interview Bite

What Are Tensor Attributes?

In PyTorch, a tensor is a multi-dimensional array that stores numerical data. Each tensor has attributes—properties that define its structure, data type, storage location, and other metadata. Understanding these attributes is crucial for debugging and efficient tensor manipulation.

Key Tensor Attributes

Attribute	Description	Example
shape	Dimensions of the tensor	torch.Size([3, 4])
dtype	Data type (e.g., float32, int64)	torch.float32
device	Where tensor is stored (CPU/GPU)	device('cuda:0')
requires_grad	Whether tensor tracks gradients (for autograd)	True/False
layout	Storage format (e.g., strided, sparse)	torch.strided
is_leaf	Whether tensor is a leaf node in computation graph	True/False

---

Code Examples: Working with Tensor Attributes

1. Creating Tensors & Checking Attributes

import torch

# Create a tensor
x = torch.tensor([[1, 2, 3], [4, 5, 6]])

# Check attributes
print("Shape:", x.shape)        # torch.Size([2, 3])
print("Data type:", x.dtype)     # torch.int64
print("Device:", x.device)      # cpu
print("Requires grad:", x.requires_grad)  # False

2. Changing Tensor Attributes

# Convert dtype
x_float = x.float()  # Changes to torch.float32
print(x_float.dtype)  

# Move tensor to GPU (if available)
x_gpu = x.to('cuda')  
print(x_gpu.device)  

# Enable gradient tracking
x.requires_grad_(True)  
print(x.requires_grad)  # True

3. Reshaping Tensors

# Reshape using .view() or .reshape()
reshaped = x.view(3, 2)  # Must maintain total elements (2x3 → 3x2)
print(reshaped.shape)     # torch.Size([3, 2])

# Flatten a tensor
flattened = x.flatten()  
print(flattened)  # tensor([1, 2, 3, 4, 5, 6])

---

Common Tensor Methods

Method	Description
.size()	Returns tensor shape (same as .shape)
.to(device/dtype)	Converts device or dtype
.cpu() / .cuda()	Moves tensor to CPU/GPU
.detach()	Removes tensor from computation graph
.numpy()	Converts to NumPy array
.item()	Extracts scalar value (for 1-element tensors)

---

Errors & Debugging Tips

Common Errors

1. Shape Mismatch
   • Occurs in operations like matrix multiplication (matmul).
   • Fix: Check tensor.shape before operations.
2. Device Mismatch
   • Error: Expected all tensors to be on the same device
   • Fix: Use .to('cuda') or .to('cpu') consistently.
3. Invalid dtype for Operation
   • Error: RuntimeError: expected scalar type Float but found Long
   • Fix: Convert dtype (e.g., .float()).

Debugging Tips

• Print tensor attributes:pythonCopyprint(f”Shape: {x.shape}, Dtype: {x.dtype}, Device: {x.device}”)
• Use torch.is_tensor() to verify object is a tensor.
• Enable CUDA error checking:pythonCopytorch.backends.cudnn.deterministic = True

---

✅ People Also Ask (FAQ)

1. What Are the Attributes of a Tensor?

Tensors have:

• Shape (dimensions)
• Data type (dtype, e.g., float32, int64)
• Device (CPU/GPU)
• Gradient tracking (requires_grad)
• Memory layout (strided or sparse)

2. What Are Tensor Properties?

Properties are attributes that define a tensor’s behavior, such as:

• Storage (storage() for raw data)
• Strides (memory jumps between elements)
• Whether it’s a leaf node (is_leaf)

3. What Are Tensors in PyTorch?

Tensors are PyTorch’s multi-dimensional arrays, similar to NumPy arrays but with GPU support and autograd capabilities.

4. What Are Tensor Values?

The actual numerical data stored in the tensor. Access using:

• .item() (for single-value tensors)
• Indexing (x[0, 1])
• .tolist() (converts to Python list)

5. How Do I Check If a Tensor Is on GPU?

print(x.is_cuda)  # True if on GPU

6. Why Does requires_grad Matter?

If True, PyTorch tracks operations for automatic differentiation (used in training neural networks).

7. How Do I Fix “RuntimeError: Expected All Tensors on Same Device”?

Ensure all tensors are on CPU or GPU:

x = x.to('cuda')
y = y.to('cuda')

---

Conclusion

Understanding tensor attributes is essential for effective PyTorch programming. Whether you’re reshaping tensors, debugging device errors, or optimizing performance, knowing shape, dtype, device, and other properties will save time and prevent bugs.