DanielAvdar/activations-plus

Collection of advanced activation functions for deep learning

What's novel

Collection of advanced activation functions for deep learning

Code Analysis

17 files read · 4 rounds

A PyTorch library providing sparse activation functions (Entmax15, Sparsemax) with custom autograd, plus a collection of simple one-liner activation function variants not natively available in PyTorch.

Strengths

Genuine, non-trivial implementations of Entmax15 and Sparsemax with proper custom autograd (forward/backward passes), comprehensive test suite including gradient checks, edge cases, and randomized testing with Hypothesis, and clean modular architecture with good documentation.

Weaknesses

README is misleading by only mentioning Entmax and Sparsemax while the package contains many more activation functions, most of the 'simple variants' are trivial one-liner wrappers with limited novelty, and Maxout/SoftClipping are essentially thin wrappers around basic PyTorch operations.

Score Breakdown

Innovation

4 (25%)

Craft

85 (35%)

Traction

11 (15%)

Scope

76 (25%)

Signal breakdown

Innovation

Not Fork+1

Code Novelty+1

Concept Novelty+1

Craft

Ci+5

Tests+8

Polish+3

Releases+4

Has License+5

Code Quality+23

Readme Quality+15

Recent Activity+7

Structure Quality+5

Commit Consistency+5

Has Dependency Mgmt+5

Traction

Forks+0

Stars+6

Hn Points+0

Watchers+3

Early Traction+0

Devto Reactions+0

Community Contribs+2

Scope

Commits+8

Languages+5

Subsystems+13

Bloat Penalty+0

Completeness+7

Contributors+6

Authored Files+15

Readme Code Match+3

Architecture Depth+7

Implementation Depth+8

Evidence

Commits

145

Contributors

Files

Active weeks

TestsCI/CDREADMELicenseContributing

Repository

Language

Python

Stars

Forks

License

MIT