@@ -166,10 +166,10 @@ The `make_tensor_like` function makes an owning Tensor of register memory with t
|
||||
Calling `print` on each of the above tensors produces similar output
|
||||
|
||||
```
|
||||
rmem_4x8_col : ptr[32b](0x7ff1c8fff820) o (_4,_8):(_1,_4)
|
||||
rmem_4x8_row : ptr[32b](0x7ff1c8fff8a0) o (_4,_8):(_8,_1)
|
||||
rmem_4x8_pad : ptr[32b](0x7ff1c8fff920) o (_4,_8):(_32,_2)
|
||||
rmem_4x8_like : ptr[32b](0x7f4158fffc60) o (_4,_8):(_8,_1)
|
||||
rmem_4x8_col : ptr[32b](0x7fff48929460) o (_4,_8):(_1,_4)
|
||||
rmem_4x8_row : ptr[32b](0x7fff489294e0) o (_4,_8):(_8,_1)
|
||||
rmem_4x8_pad : ptr[32b](0x7fff489295e0) o (_4,_8):(_32,_2)
|
||||
rmem_4x8_like : ptr[32b](0x7fff48929560) o (_4,_8):(_8,_1)
|
||||
```
|
||||
|
||||
and we can see that each pointer address is unique indicating that each `Tensor` is a unique array-like allocation.
|
||||
@@ -195,7 +195,7 @@ For example, we can read and write to `Tensor`s using natural coordinates, using
|
||||
|
||||
```c++
|
||||
Tensor A = make_tensor<float>(Shape <Shape < _4,_5>,Int<13>>{},
|
||||
Stride<Stride<_12,_1>,_64>{});
|
||||
Stride<Stride<_12,_1>, _64>{});
|
||||
float* b_ptr = ...;
|
||||
Tensor B = make_tensor(b_ptr, make_shape(13, 20));
|
||||
|
||||
@@ -317,7 +317,7 @@ Another common partitioning strategy is called a thread-value partitioning. In t
|
||||
// to 1D coordinates within a 4x8 tensor
|
||||
// (T8,V4) -> (M4,N8)
|
||||
auto tv_layout = Layout<Shape <Shape <_2,_4>,Shape <_2, _2>>,
|
||||
Stride<Stride<_8,_1>,Stride<_4,_16>>>{}; // (8,4)
|
||||
Stride<Stride<_8,_1>,Stride<_4,_16>>>{}; // (8,4)
|
||||
|
||||
// Construct a 4x8 tensor with any layout
|
||||
Tensor A = make_tensor<float>(Shape<_4,_8>{}, LayoutRight{}); // (4,8)
|
||||
|
||||
@@ -195,7 +195,7 @@ As is evident, these smem layouts can be almost anything. Inside the kernel, the
|
||||
CUTE_STATIC_ASSERT_V(size<1>(BSmemLayout{}) == size<2>(cta_tiler)); // BLK_K
|
||||
```
|
||||
|
||||
Use of static layouts has a few advantages.
|
||||
Use of static layouts has a few advantages.
|
||||
* Static layouts let us statically allocate shared memory as shown below.
|
||||
* Static layouts are often more efficient and allow CuTe to dispatch to optimized implementations.
|
||||
* Static layouts makes it easier to prove correctness of the algorithm and provide checks like the above -- the smem layout sizes are the same as the CTA tile sizes.
|
||||
@@ -227,7 +227,7 @@ if (thread0()) {
|
||||
```
|
||||
This would work, but we have lots of threads to use inside this CTA, so let's use them!
|
||||
|
||||
If we partition the two tiles of data across the threads in the CTA, then each thread can copy its own subtensor of data. There are lots of ways this partitioning could occur, however.
|
||||
If we partition the two tiles of data across the threads in the CTA, then each thread can copy its own subtensor of data. There are lots of ways this partitioning could occur, however.
|
||||
|
||||
The `gemm_nt` function defines two layouts of *threads* as
|
||||
```c++
|
||||
@@ -295,7 +295,7 @@ if (thread0()) {
|
||||
```
|
||||
This would work, but we have lots of threads to use inside this CTA, so let's use them!
|
||||
|
||||
If we partition the output tile `gC` across the threads in the CTA, then each thread can compute its own subtensor. There are lots of ways this partitioning could occur, however.
|
||||
If we partition the output tile `gC` across the threads in the CTA, then each thread can compute its own subtensor. There are lots of ways this partitioning could occur, however.
|
||||
|
||||
The `gemm_nt` and `gemm_tn` functions define one more layout of *threads*:
|
||||
```cpp
|
||||
@@ -332,7 +332,7 @@ These thread layouts are then used to partition the tiles of data in global memo
|
||||
CUTE_STATIC_ASSERT_V(size<1>(tCrC) == size<0>(tCsB)); // THR_N
|
||||
CUTE_STATIC_ASSERT_V(size<1>(tCsA) == size<1>(tCsB)); // BLK_K
|
||||
```
|
||||
where we've used the same projection-style interface to avoid applying the `N`-mode of `tC` to the `(BLK_M,BLK_K)` shape of `sA` and avoid applying the `M`-mode of `tC` to the `(BLK_N,BLK_K)` shape of `sB`.
|
||||
where we've used the same projection-style interface to avoid applying the `N`-mode of `tC` to the `(BLK_M,BLK_K)` shape of `sA` and avoid applying the `M`-mode of `tC` to the `(BLK_N,BLK_K)` shape of `sB`.
|
||||
|
||||
<p align="center">
|
||||
<img src="../../images/cute/tC_partitioning.png" alt="tC_partitioning.png" height="300"/>
|
||||
|
||||
Reference in New Issue
Block a user