leavelet/cutlass
1
0
Fork 0
Code Issues Pull Requests Actions 29 Packages Projects Releases Wiki Activity

[hardswish] correct implmentation (#403)

Browse Source 
* [hardswish] correct implmentation

* seems working

* hardswish fp32/fp16x2 optimization

* [relu] half2 support

* add relu0; add multiply_add_relu0;

* cleanup

Co-authored-by: Bing Xu <bingxu@fb.com>
Co-authored-by: Haicheng Wu <haichengw@nvidia.com>
This commit is contained in:
Bing Xu
2022-02-09 14:28:53 -05:00
committed by GitHub
co-authored by Bing Xu Haicheng Wu
parent 8a951b2940
commit d0d941efc7
12 changed files with 1315 additions and 235 deletions
Download Patch File Download Diff File Expand all files Collapse all files
include/cutlass/epilogue/thread/linear_combination_sigmoid.h
+4 -2
View File
@@ -1,5 +1,5 @@
/***************************************************************************************************
* Copyright (c) 2017-2021, NVIDIA CORPORATION. All rights reserved.
* Copyright (c) 2017-2022, NVIDIA CORPORATION. All rights reserved.
*
* Redistribution and use in source and binary forms, with or without modification, are permitted
* provided that the following conditions are met:
@@ -51,10 +51,12 @@ template <
///< but we use 64 or 32 sometimes when there are not enough data to store
typename ElementAccumulator_ = ElementOutput_, ///< Accumulator data type
typename ElementCompute_ = ElementOutput_, ///< Data type used to compute linear combination
ScaleType::Kind Scale = ScaleType::Default, ///< Control Alpha and Beta scaling
FloatRoundStyle Round = FloatRoundStyle::round_to_nearest
>
using LinearCombinationSigmoid = LinearCombinationGeneric<Sigmoid, ElementOutput_, Count, ElementAccumulator_,
ElementCompute_, Round, true>;
ElementCompute_, Scale, Round, true>;
/////////////////////////////////////////////////////////////////////////////////////////////////
} // namespace thread