黑盒模型?顶级对冲基金Jane Street带你可视化神经网络
神经网络是现代人工智能的核心技术之一,但它们常常被认为是黑盒函数逼近器,由于其内部结构复杂,难以解释。幸运的是,理论工具的引入帮助我们更好地理解神经网络的行为,尤其是分段线性这一特性。本文将以更详细的方式介绍和可视化神经网络中的分段线性现象。
Jane Street 简介
Jane Street是一家全球知名的量化交易公司,专注于使用最先进的技术和算法进行金融市场交易。公司以使用函数式编程语言OCaml闻名,并且在技术社区中以创新的技术实践和开源项目作出了卓越贡献。Jane Street的技术团队不仅仅是支持交易的工具开发者,还在诸多前沿技术领域开展了深入研究。
什么是分段线性?
神经网络的一个核心特点是其分段线性性质。分段线性是指一个函数可以分解为多个线性部分,尽管这些部分的组合可能并不是线性的。这个特性帮助神经网络能够学习和逼近非线性问题中的复杂函数。
ReLU激活函数
一个常见的激活函数是ReLU(Rectified Linear Unit),它可以被看作由两个线性部分组成:当输入为正时输出等于输入;当输入为负时输出为零。因此,ReLU的激活函数可以被描述为以下形式:
这就是典型的分段线性函数,它在原点处连接两部分。

基础神经网络架构
我们讨论的神经网络架构是非常基础但功能强大的。其核心思想是将线性层与ReLU激活函数交替使用。这种结构在许多深度学习任务中被广泛应用。
首先,我们来看一个单层神经网络的例子。它有两个输入和一个输出神经元,输出神经元使用ReLU激活函数。输入分别位于x轴和y轴,而输出位于z轴。

ReLU的激活与关闭
在这个网络中,ReLU函数会在输入空间中划分出激活和非激活区域。在图中,左半部分ReLU关闭,输出为零;右半部分ReLU开启,输出等于输入。
神经网络只能学习连续分段线性函数
一个重要的限制是,神经网络只能学习连续的分段线性函数。这意味着网络无法学习不连续的函数。例如,以下是一个无法被神经网络学习的函数,因为它的两部分在边界处并不相连。

增加输出神经元
接下来,我们将模型复杂度提升一步。通过增加输出神经元的数量,可以看到网络划分的区域更多了。假设我们将输出神经元增加到8个,那么网络将形成更多的分段区域。每一个区域对应于某些ReLU被激活而其他未激活的子集,这就是所谓的激活模式。
激活模式的组合
理论上,8个神经元应该有2^8 = 256种激活模式。然而,由于我们限制在二维空间中,实际上只有37种模式是可行的,其中32种可以在下图中可见。这些结构形成了一个复杂的几何结构,称为多面体复合体。

以下是同一个几何结构的鸟瞰视角,它以逆时针旋转90度呈现:

添加第二层神经元
为了增加模型的复杂性,我们可以再添加一层神经元。这一层同样包含8个神经元,并且使用ReLU激活函数。为了在图中与第一层的线条区分,我们将第二层的线条绘制得更细。
二层网络中的线性组合
因为线性函数的组合仍然是线性的,所以在第一层划分出的每个区域内,第二层的线条依然是直线。然而,当这些线条碰到分段边界时,线性函数发生变化,产生了弯折。若某个激活模式不可行,线条将会终止。
输出值的可视化
随着网络层次的增加,圆形结构开始出现,并在接下来的层次中进一步细化。输出值较高的区域用较亮的颜色表示,输出值较低的区域用较暗的颜色表示,这使得我们可以更加直观地理解神经网络的输出。
最终输出结果
以下是经过训练后的神经网络的最终输出结果。图中展示了神经网络输出的3D视角:

神经网络权重变化的演变
我们不仅可以研究单个经过训练的神经网络,还可以通过可视化整个多面体复合体来探索神经网络在训练过程中权重变化的演变。
首先,我们从一个随机初始化权重的未训练网络开始。随着训练的进行,网络逐步生成一些可识别的形状。未训练的权重会将输入平面划分为少量多边形,而随着训练的进行,更多的多边形会形成,逐渐表现出复杂的模式。
额外注释
- 1. 激活函数选择:尽管我们在解释中使用的是ReLU,但实际可视化中使用的是LeakyReLU(0.02),因为小型ReLU网络在训练时容易陷入局部极小值。
- 2. 输出维度:网络的输出现在是8维的,但为了方便可视化,我们选择了任意的线性投影,使得我们可以在三维空间中呈现结果。
- 3. 几何形状的准确性:技术上,这些区域中有些是无界的,因此并不是严格意义上的多边形。更准确的术语是**“半空间交集”**,即这些区域的边界延伸至无穷远。
previous_layer_regions = [euclidean_space]
for each layer:
regions = []
for parent_region in previous_layer_regions:
for active_neurons in power_set_of_neurons:
region = solve_constraints_for_activation_pattern()
if region is feasible:
regions.append(region)
previous_layer_regions = regions

This message is used to verify that this feed (feedId:73016807219321856) belongs to me (userId:72982329363080192). Join me in enjoying the next generation information browser https://follow.is.
Yesterday, while I was at work, my sister stole my iphone andd
tested to see if it can survive a 25 foot drop, just so she can be a youtybe sensation. My apple ipad is noow broken and she
has 83 views. I know this is completely off toipic but I had to share it
with someone! https://hot-fruits-Glassi.blogspot.com/2025/08/hot-fruitsslot.html
An outstanding share! I’ve jhst forwarded this onto a colleague who haas been conducting a little research on this.
And he in fact ordered me lunch simply because I discovcered it for him…
lol. So allow me to reword this…. Thanks forr the meal!!
But yeah, thsnks for spending time to talk about tthis matter here on your website. https://skillnaukri.com/employer/tonybet/
india pharmacy: IndiavaMeds – indian pharmacy
https://aeromedsrx.com/# AeroMedsRx
http://aeromedsrx.com/# Viagra online price
AeroMedsRx [url=https://aeromedsrx.xyz/#]viagra without prescription[/url] AeroMedsRx
order Kamagra discreetly: kamagra oral jelly – Blue Wave Meds
https://everameds.com/# buy cialis pill
https://aeromedsrx.com/# AeroMedsRx
EveraMeds: п»їcialis generic – Generic Tadalafil 20mg price
https://bluewavemeds.com/# trusted Kamagra supplier in the US
Buy generic 100mg Viagra online: AeroMedsRx – AeroMedsRx
http://bluewavemeds.com/# Blue Wave Meds
Cialis 20mg price in USA [url=https://everameds.com/#]Buy Tadalafil 5mg[/url] EveraMeds
https://bluewavemeds.xyz/# kamagra oral jelly
Generic Cialis without a doctor prescription [url=https://everameds.xyz/#]EveraMeds[/url] EveraMeds
kamagra: trusted Kamagra supplier in the US – trusted Kamagra supplier in the US
AeroMedsRx: Sildenafil Citrate Tablets 100mg – AeroMedsRx
https://aeromedsrx.xyz/# buy Viagra online
order Kamagra discreetly [url=https://bluewavemeds.com/#]online pharmacy for Kamagra[/url] Blue Wave Meds
AeroMedsRx: viagra without prescription – generic sildenafil
EveraMeds: Tadalafil price – EveraMeds
EveraMeds [url=https://everameds.com/#]EveraMeds[/url] Tadalafil Tablet
cheapest cialis: Cialis over the counter – EveraMeds
buy Viagra over the counter: AeroMedsRx – AeroMedsRx
online pharmacy for Kamagra [url=https://bluewavemeds.xyz/#]kamagra[/url] kamagra
https://aeromedsrx.xyz/# AeroMedsRx
https://bluewavemeds.com/# order Kamagra discreetly
buy viagra here: cheapest viagra – sildenafil 50 mg price
online pharmacy for Kamagra [url=http://bluewavemeds.com/#]trusted Kamagra supplier in the US[/url] trusted Kamagra supplier in the US
https://aeromedsrx.xyz/# buy viagra here
Tadalafil price: EveraMeds – Cialis 20mg price
Blue Wave Meds: kamagra – trusted Kamagra supplier in the US
Cheap Cialis [url=https://everameds.com/#]EveraMeds[/url] EveraMeds
http://aeromedsrx.com/# AeroMedsRx
EveraMeds: Buy Tadalafil 10mg – Cialis 20mg price
EveraMeds [url=http://everameds.com/#]EveraMeds[/url] EveraMeds
https://bluewavemeds.xyz/# online pharmacy for Kamagra
AeroMedsRx: AeroMedsRx – Viagra generic over the counter
online pharmacy for Kamagra [url=https://bluewavemeds.com/#]kamagra oral jelly[/url] online pharmacy for Kamagra
AeroMedsRx: Sildenafil Citrate Tablets 100mg – cheap viagra
fast delivery Kamagra pills: fast delivery Kamagra pills – order Kamagra discreetly
Sildenafil Citrate Tablets 100mg [url=https://aeromedsrx.xyz/#]buy Viagra online[/url] viagra without prescription
EveraMeds: Cialis 20mg price – Tadalafil Tablet
online pharmacy for Kamagra [url=https://bluewavemeds.xyz/#]kamagra oral jelly[/url] online pharmacy for Kamagra
https://bluewavemeds.com/# fast delivery Kamagra pills
EveraMeds [url=https://everameds.com/#]EveraMeds[/url] Buy Tadalafil 5mg
buy Viagra over the counter: AeroMedsRx – cheap viagra
http://everameds.com/# EveraMeds
buy cialis pill: EveraMeds – Generic Cialis price
http://bluewavemeds.com/# buy Kamagra online
trusted Kamagra supplier in the US: kamagra – buy Kamagra online
kamagra oral jelly [url=https://bluewavemeds.com/#]BlueWaveMeds[/url] buy Kamagra online
fast delivery Kamagra pills: kamagra – BlueWaveMeds
AeroMedsRx: AeroMedsRx – AeroMedsRx
EveraMeds [url=http://everameds.com/#]EveraMeds[/url] EveraMeds
http://aeromedsrx.com/# Viagra online price
sildenafil over the counter: AeroMedsRx – Generic Viagra online
Sildenafil 100mg price [url=https://aeromedsrx.xyz/#]AeroMedsRx[/url] AeroMedsRx
https://everameds.com/# EveraMeds
Buy Tadalafil 20mg: Tadalafil price – Cialis 20mg price
viagra without prescription [url=https://aeromedsrx.xyz/#]AeroMedsRx[/url] AeroMedsRx
https://aeromedsrx.com/# AeroMedsRx
EveraMeds: EveraMeds – EveraMeds
kamagra oral jelly: buy Kamagra online – trusted Kamagra supplier in the US
BlueWaveMeds [url=https://bluewavemeds.com/#]online pharmacy for Kamagra[/url] BlueWaveMeds
https://aeromedsrx.com/# best price for viagra 100mg
kamagra: order Kamagra discreetly – Blue Wave Meds
Blue Wave Meds [url=http://bluewavemeds.com/#]fast delivery Kamagra pills[/url] Blue Wave Meds
http://everameds.com/# Generic Tadalafil 20mg price
http://isoindiapharm.com/# buy medicines online in india
IsoIndiaPharm: IsoIndiaPharm – indian pharmacy online
safe reliable canadian pharmacy https://mhfapharm.xyz/# MHFA Pharm
order antibiotics from mexico: UvaPharm – online mexican pharmacies
https://uvapharm.com/# Uva Pharm
canadian drugstore online: MHFA Pharm – reputable canadian pharmacy
canadian neighbor pharmacy https://mhfapharm.com/# MhfaPharm
MhfaPharm: reputable canadian online pharmacy – MHFA Pharm
india online pharmacy [url=http://isoindiapharm.com/#]best india pharmacy[/url] IsoIndiaPharm
https://isoindiapharm.xyz/# Iso Pharm
pharmacies in canada that ship to the us http://uvapharm.com/# pharmacys in mexico
mexico medicine: UvaPharm – mexico pet pharmacy
https://mhfapharm.com/# MhfaPharm
canadian pharmacy service https://mhfapharm.com/# MhfaPharm
UvaPharm: mexicanrxpharm – Uva Pharm
https://isoindiapharm.com/# reputable indian pharmacies
canadianpharmacymeds http://isoindiapharm.com/# Iso Pharm
IsoIndiaPharm: IsoIndiaPharm – buy medicines online in india
Uva Pharm: mexican pharmacies – phentermine in mexico pharmacy
online pharmacy in mexico [url=http://uvapharm.com/#]UvaPharm[/url] medicine mexico
MHFA Pharm: MHFA Pharm – MhfaPharm
https://uvapharm.com/# UvaPharm
canadian pharmacy prices https://isoindiapharm.com/# IsoIndiaPharm
Online medicine order: IsoIndiaPharm – IsoIndiaPharm
MhfaPharm [url=http://mhfapharm.com/#]MHFA Pharm[/url] MhfaPharm
canadian pharmacy http://uvapharm.com/# UvaPharm
Iso Pharm: IsoIndiaPharm – Iso Pharm
https://mhfapharm.xyz/# canada pharmacy online
real canadian pharmacy https://uvapharm.com/# UvaPharm
best india pharmacy: IsoIndiaPharm – Iso Pharm
canadian neighbor pharmacy http://isoindiapharm.com/# Iso Pharm
Uva Pharm: Uva Pharm – Uva Pharm
IsoIndiaPharm [url=https://isoindiapharm.xyz/#]IsoIndiaPharm[/url] indian pharmacies safe
legit canadian pharmacy https://mhfapharm.com/# MhfaPharm
https://isoindiapharm.xyz/# india pharmacy
https://isoindiapharm.com/# Iso Pharm
Ucla Metformin [url=https://uclametformin.com/#]UclaMetformin[/url] UclaMetformin
mail order pharmacy india http://socalabortionpill.com/# buy cytotec online
UclaMetformin: UclaMetformin – 143 metformin
Socal Abortion Pill: buy misoprostol over the counter – Socal Abortion Pill
online shopping pharmacy india https://pmaivermectin.com/# PmaIvermectin
UclaMetformin: Ucla Metformin – metformin 1 mg
BswFinasteride [url=http://bswfinasteride.com/#]BSW Finasteride[/url] BSW Finasteride
SocalAbortionPill: Socal Abortion Pill – buy abortion pills
world pharmacy india http://socalabortionpill.com/# buy abortion pills
PmaIvermectin: PmaIvermectin – PmaIvermectin
online shopping pharmacy india http://pmaivermectin.com/# ivermectin 8000 mcg
metformin prescription online: metformin 500mg tablets price in india – metformin 1000 mg
PmaIvermectin: ivermectin for kids – PmaIvermectin
cost propecia no prescription [url=https://bswfinasteride.xyz/#]BswFinasteride[/url] buy cheap propecia without rx
best india pharmacy https://pmaivermectin.com/# durvet ivermectin paste
SocalAbortionPill: Misoprostol 200 mg buy online – buy cytotec over the counter
metformin.com: UclaMetformin – metformin 500 mg
mail order pharmacy india http://uclametformin.com/# UclaMetformin
top 10 online pharmacy in india https://bswfinasteride.xyz/# BswFinasteride
BSW Finasteride: BSW Finasteride – BswFinasteride
PmaIvermectin [url=http://pmaivermectin.com/#]PmaIvermectin[/url] ivermectin dose for scabies
PMA Ivermectin: PmaIvermectin – how to dose ivermectin for humans
online pharmacy india http://uclametformin.com/# Ucla Metformin
buy abortion pills: buy cytotec – Socal Abortion Pill
metformin medicine: metformin hcl 500 mg – UclaMetformin
canadian pharmacies without prescriptions http://neokamagra.com/# NeoKamagra
https://neokamagra.com/# buy kamagra online usa
super kamagra: Neo Kamagra – Kamagra 100mg price
Dmu Cialis [url=http://dmucialis.com/#]DmuCialis[/url] Dmu Cialis
buying drugs canada http://muscpharm.com/# Musc Pharm
DmuCialis: Dmu Cialis – DmuCialis
Musc Pharm: canadian pharmacies top best – canadian neighbor pharmacy legit
NeoKamagra [url=https://neokamagra.com/#]NeoKamagra[/url] NeoKamagra
canadian pharma companies https://muscpharm.xyz/# MuscPharm
https://muscpharm.com/# Musc Pharm
NeoKamagra [url=http://neokamagra.com/#]sildenafil oral jelly 100mg kamagra[/url] Neo Kamagra
DmuCialis: Dmu Cialis – Buy Tadalafil 10mg
Dmu Cialis: DmuCialis – DmuCialis
canadian pharmaceutical companies that ship to usa http://muscpharm.com/# MuscPharm
safe online pharmacies http://dmucialis.com/# DmuCialis
NeoKamagra [url=http://neokamagra.com/#]Neo Kamagra[/url] buy kamagra online usa
canada pharmacy world: online canadian pharmacy no prescription – Musc Pharm
https://neokamagra.com/# Neo Kamagra
DmuCialis: Dmu Cialis – Dmu Cialis
canadian meds https://dmucialis.com/# DmuCialis
canadiandrugstore.com [url=https://muscpharm.com/#]MuscPharm[/url] Musc Pharm
DmuCialis: Dmu Cialis – Dmu Cialis
reputable canadian pharmacy online https://neokamagra.xyz/# NeoKamagra
NeoKamagra [url=https://neokamagra.com/#]Neo Kamagra[/url] NeoKamagra
https://neokamagra.xyz/# NeoKamagra
canadian pharmacies mail order https://neokamagra.xyz/# п»їkamagra
MuscPharm [url=https://muscpharm.com/#]MuscPharm[/url] Musc Pharm
Musc Pharm: Musc Pharm – MuscPharm
compare medication prices http://muscpharm.com/# Musc Pharm
Musc Pharm [url=http://muscpharm.com/#]Musc Pharm[/url] Musc Pharm
mexican pharmacy online: canada pharmacy not requiring prescription – real canadian pharmacy
compare prescription drug prices https://dmucialis.xyz/# DmuCialis
http://neokamagra.com/# Neo Kamagra
Kamagra 100mg price [url=https://neokamagra.xyz/#]NeoKamagra[/url] NeoKamagra
Kamagra 100mg price: Kamagra tablets – NeoKamagra
discount canadian drugs https://muscpharm.xyz/# Musc Pharm
NeoKamagra [url=https://neokamagra.com/#]sildenafil oral jelly 100mg kamagra[/url] Neo Kamagra
Neo Kamagra: Neo Kamagra – sildenafil oral jelly 100mg kamagra
great canadian pharmacy http://neokamagra.com/# NeoKamagra
http://neokamagra.com/# buy kamagra online usa
Kamagra tablets [url=http://neokamagra.com/#]buy kamagra online usa[/url] super kamagra
Musc Pharm: trusted canadian pharmacy – MuscPharm
cheap canadian drugs https://muscpharm.xyz/# amazon pharmacy drug prices
Dmu Cialis: Dmu Cialis – Dmu Cialis
MuscPharm: MuscPharm – rx canada
online drugstore reviews https://muscpharm.xyz/# MuscPharm
online pharmacies canada reviews: MuscPharm – recommended online pharmacies
NeoKamagra: sildenafil oral jelly 100mg kamagra – Neo Kamagra
https://dmucialis.com/# DmuCialis
prescription drug price check https://neokamagra.com/# NeoKamagra
sun cruz casino
References:
https://mm.gd/read-blog/26572_today-039-s-interactive-breaking-news-quiz.html
Neo Kamagra: Neo Kamagra – Neo Kamagra
Dmu Cialis: Dmu Cialis – DmuCialis
canadian medicine https://neokamagra.com/# NeoKamagra
Cialis without a doctor prescription: Generic Cialis price – DmuCialis
CorPharmacy: Cor Pharmacy – online pharmacy non prescription drugs
ViagraNewark [url=http://viagranewark.com/#]Sildenafil Citrate Tablets 100mg[/url] Order Viagra 50 mg online
safe canadian internet pharmacies http://edpillsafib.com/# EdPillsAfib
ViagraNewark: buy Viagra online – Buy Viagra online cheap
CorPharmacy: Cor Pharmacy – CorPharmacy
Ed Pills Afib [url=https://edpillsafib.com/#]Ed Pills Afib[/url] Ed Pills Afib
canadian pharmacies recommended https://corpharmacy.xyz/# CorPharmacy
Cor Pharmacy: Cor Pharmacy – CorPharmacy
https://viagranewark.xyz/# viagra canada
Ed Pills Afib [url=http://edpillsafib.com/#]Ed Pills Afib[/url] best ed medication online
internet pharmacies https://corpharmacy.com/# Cor Pharmacy
Viagra Newark: ViagraNewark – ViagraNewark
EdPillsAfib [url=https://edpillsafib.xyz/#]EdPillsAfib[/url] EdPillsAfib
http://edpillsafib.com/# EdPillsAfib
ViagraNewark [url=http://viagranewark.com/#]Viagra tablet online[/url] ViagraNewark
prescription drug price check https://edpillsafib.com/# erectile dysfunction online
sildenafil online: Viagra Newark – Viagra Newark
buy Viagra online [url=http://viagranewark.com/#]Viagra Newark[/url] ViagraNewark
canadian pharmacy phone number: Cor Pharmacy – CorPharmacy
Viagra Newark: ViagraNewark – Cheapest Sildenafil online
Viagra Newark: Buy generic 100mg Viagra online – generic sildenafil