]irBSSKrSSKrSSKrSSKrSSKJr SSKJr SSK J r SSK J r J r SSKJr "SS \R 5rS rS r"S S \R 5r"SS\R 5r"SS\R 5rg)N)nn) functional) capture_init) center_trimunfold) LayerScalec6^\rSrSrSrSU4SjjrSrSrU=r$)BLSTMz BiLSTM with same hidden units as input dim. If `max_steps` is not None, input will be splitting in overlapping chunks and the LSTM applied separately on each chunk. c>[TU]5 Ub US-S:XdeX0l[R"SX!US9Ul[R "SU-U5UlX@lg)NrT) bidirectional num_layers hidden_size input_size) super__init__ max_stepsrLSTMlstmLinearlinearskip)selfdimlayersrr __class__s J/mnt/rpi/tmp/demucs-venv-sys/lib/python3.13/site-packages/demucs/demucs.pyrBLSTM.__init__sZ  IMQ$666"GG$6_bc iiC-  c NURup#nUnSnURbaX@R:aRURnUS-n[XU5n U RSn SnU RSSSS5R SX75nURSSS5nUR U5SnUR U5nURSSS5nU(a/n UR USUW5n WS-n [W 5Hpn U S:Xa!U RU SS2U SS2SU *245 M*XS- :Xa U RU SS2U SS2U S245 MRU RU SS2U SS2X*245 Mr [R"U S5n U SSU24n U nUR(aX-nU$) NFrTrr.) shaperrpermutereshaperrrangeappendtorchcatr)rxBCTyframedwidthstrideframesnframesoutlimitks r forward BLSTM.forward"s''a  >> %!nn*<NNEaZFAf-Fll1oGFq!Q*222q@A IIaA  IIaLO KKN IIaA  CYYq"a/FaKE7^6JJvaAww&678A+%JJvaAuvo67JJvaAuV|&;<= $))C$Cc2A2g,CA 99Ar")rrrr)rNF) __name__ __module__ __qualname____firstlineno____doc__rr:__static_attributes__ __classcell__rs@r r r s !!r"r cURR5R5nX!- S-nUR=RU-slURb UR=RU-slgg)zTRescale initial weight scale. It is unclear why it helps but it certainly does. ?N)weightstddetachdatabias)conv referencerGscales r rescale_convrNFs_ ++//  " " $C _s "EKK yy %r"cUR5H]n[U[R[R[R [R 45(dMR[X!5 M_ gN)modules isinstancerConv1dConvTranspose1dConv2dConvTranspose2drN)modulerLsubs r rescale_modulerYPsE~~ cBIIr'9'9299bFXFXY Z Z  ( r"c P^\rSrSrSrS S\S\S\S\4U4SjjjrSrS r U=r $) DConvVz New residual branches in each encoder layer. This alternates dilated convolutions, potentially with LSTMs and attention. Also before entering each residual branch, dimension is projected on a smaller subspace, e.g. of dim `channels // compress`. channelscompressdepthinitc L>[TU]5 U S-S:XdeXlX l[ U5UlUS:n Sn U(aSn [ X- 5nU (a[RnO[Rn[R"/5Ul [UR 5HnU (aSU-OSnUU S--n[R"XU UUS9U "U5U"5[R"USU-S5U "SU-5[R"S5[X5/nU(aUR!S[#XUS95 U (aUR!S[%USS S S 95 [R&"U6nURR)U5 M g ) ax Args: channels: input/output channels for residual branch. compress: amount of channel compression inside the branch. depth: number of layers in the residual branch. Each layer has its own projection, and potentially LSTM and attention. init: initial scale for LayerNorm. norm: use GroupNorm. attn: use LocalAttention. heads: number of heads for the LocalAttention. ndecay: number of decay controls in the LocalAttention. lstm: use LSTM. gelu: Use GELU activation. kernel: kernel size for the (dilated) convolutions. dilate: if true, use dilation, increasing with the depth. rrrc,[R"5$rPrIdentityds r  DConv.__init__..ys BKKMr"c0[R"SU5$)Nrr GroupNormres r rgrh{s Q 2r")dilationpaddingr$)headsndecayT)rrrN)rrr]r^absr_intrGELUReLU ModuleListrr)rSGLUr insert LocalStater Sequentialr*)rr]r^r_r`normattnrnrorgelukerneldilatenorm_fnhiddenactrfrlrmmodslayerrs r rDConv.__init__]s[( zQ  Z * 2GX() ''C''CmmB' tzz"A!'qAvQH&A+.G (FXwW &!h,2H %rvvay8* D Az&fMN AuVA4PQMM4(E KK  u %#r"c@URH nX"U5-nM U$rP)r)rr-rs r r: DConv.forwards"[[EE!H A!r")r]r^r_r) rr-C6?TFrrFTr$T) r<r=r>r?r@rrfloatrr:rArBrCs@r r[r[VsG Z^LP"&8&8&8&#8&QV8&8&tr"r[c J^\rSrSrSrS S\S\S\S\4U4SjjjrSrS rU=r $) rxzLocal state allows to have attention based only on data (no positional embedding), but while setting a constraint on the time window (e.g. decaying penalty term). Also a failed experiments with trying to provide some frequency based attention. r]rnnfreqsroc>[TU]5 X-S:XdX45eX lX0lX@l[ R "XS5Ul[ R "XS5Ul[ R "XS5Ul U(a[ R "XU-S5Ul U(a[ R "XU-S5Ul URR=RS-sl URRceSURRRSS&[ R "XU--US5Ulg)Nrrg{Gz?)rrrnrrorrScontentquerykey query_freqs query_decayrFrIrJproj)rr]rnrrors r rLocalState.__init__s  1$7x&77$   yyQ7 YYx15 99X3 !yy6>1ED  !yy6>1ED     # # ( (D 0 (##((4 44,.D   ! ! & &q )IIh71E r"cURup#nURn[R"XARUR S9nUSS2S4USSS24- nUR U5RX%SU5nURU5RX%SU5n [R"SX5n XRSS--n UR(a[R"SURS-URUR S9n [R"S[R-U-U RSSS5- 5n URU5RX%SU5URS-- n U [R"SX5- n UR(a[R"SURS-URUR S9nUR!U5RX%SU5n[R""U5S- nURSSS5*UR%5-URS-- nU [R"SUU5- n U R'[R("XJR[R*S9S5 [R,"U SS 9nUR/U5RX%SU5n[R"S UU5nUR(a1[R"S UW 5n[R0"UU/S5nUR3USU5nXR5U5-$) N)devicedtyper%zbhct,bhcs->bhtsrrErzfts,bhfs->bhtsirzbhts,bhct->bhcszbhts,fts->bhfs)r&rnr+arangerrrviewreinsumrcosmathpirrorsigmoidrq masked_fill_eyeboolsoftmaxrr,r(r)rr-r.r/r0rnindexesdeltaquerieskeysdotsperiods freq_kernelfreq_qdecaysdecay_q decay_kernelweightsrresulttime_sigs r r:LocalState.forwards''a ,,qA4 747#33**Q-$$Qr15xx{"a0||-t= 1 s"" ;;ll1dkkAoahhaggVG))AK%$7',,r1a:P$PQK%%a(--aA>PSASSF ELL!1;G GD ;;\\!T[[1_QXXQWWUF&&q)..qQ?GmmG,q0G#[[Q22UYY[@4;;PSCSSL ELL!1<I ID %))AkkLdS--!,,,q/&&qQ7/'B ;;||$4g{KHYY115F2q)99V$$$r") rrnrrorrrrr)rrr) r<r=r>r?r@rrrr:rArBrCs@r rxrxs> FFSFcFsFF&"%"%r"rxc^\rSrSr\SU4Sjj5rSrSrSU4SjjrSr U=r $) Demucsc >^[T&U]5 X lXlXlXlXlXPlUUlX0l UUl UUl UUl [R"5Ul[R"5Ul[R"5UlU (a[R$"SS9nSnO[R&"5nSnU (a[R(nO[R&nUnSn[+U5GHnSn UU :aU4Sjn /n!U![R,"UX8U 5U "U5U"5/- n!UU:n"UU:n#US-(aU![/UUUUU"U#S9/- n!U(a*U![R,"UUU-S5U "UU-5U/- n!URR1[R2"U!65 /n$US:aUn%O[5UR5U-n%U(a/U$[R,"UUU-SU -S-U S9U "UU-5U/- n$US-(aU$[/UUUUU"U#S9/- n$U$[R6"UU%XUS9/- n$US:aU$U "U%5U"5/- n$UR R9S[R2"U$65 Un[;XC-5nGM UnU(a[=X75UlOS UlU(a [AUUS 9 g g ) a2 Args: sources (list[str]): list of source names audio_channels (int): stereo or mono channels (int): first convolution channels depth (int): number of encoder/decoder layers growth (float): multiply (resp divide) number of channels by that for each layer of the encoder (resp decoder) depth (int): number of layers in the encoder and in the decoder. rewrite (bool): add 1x1 convolution to each layer. lstm_layers (int): number of lstm layers, 0 = no lstm. Deactivated by default, as this is now replaced by the smaller and faster small LSTMs in the DConv branches. kernel_size (int): kernel size for convolutions stride (int): stride for convolutions context (int): kernel size of the convolution in the decoder before the transposed convolution. If > 1, will provide some context from neighboring time steps. gelu: use GELU activation function. glu (bool): use glu instead of ReLU for the 1x1 rewrite conv. norm_starts: layer at which group norm starts being used. decoder layers are numbered in reverse order. norm_groups: number of groups for group norm. dconv_mode: if 1: dconv in encoder only, 2: decoder only, 3: both. dconv_depth: depth of residual DConv branch. dconv_comp: compression of DConv branch. dconv_attn: adds attention layers in DConv branch starting at this layer. dconv_lstm: adds a LSTM layer in DConv branch starting at this layer. dconv_init: initial scale for the DConv branch LayerScale. normalize (bool): normalizes the input audio on the fly, and scales back the output by the same amount. resample (bool): upsample x2 the input and downsample /2 the output. rescale (float): rescale initial weights of convolutions to get their standard deviation closer to `rescale`. samplerate (int): stored as meta information for easing future evaluations of the model. segment (float): duration of the chunks of audio to ideally evaluate the model on. This is used by `demucs.apply.apply_model`. rrrrc,[R"5$rPrcres r rg!Demucs.__init__..Fs  r"c2>[R"TU5$rPrj)rf norm_groupss r rgrHsBLLa$@r")r_r`r^r{r)rmN)rL)!rraudio_channelssources kernel_sizecontextr4r_resampler] normalize sampleratesegmentrruencoderdecoder skip_scalesrvrtrsr)rSr[r*rylenrTrwrrr rrY)'rrrr]growthr_rewrite lstm_layersrr4rr|glu norm_startsr dconv_mode dconv_depth dconv_comp dconv_attn dconv_lstm dconv_initrrrescalerr activationch_scaleact2 in_channelsrmindexrencoder{rdecode out_channelsrs' ` r rDemucs.__init__sX , &      "$ }} }} ==? AJHJH 77D77D$ 5\E-G #@F  +xfE! F J&DJ&DA~5:*44dLMMIIh8(;Q?Hx/0*>> LL   v 6 7Fqy* "4<<0>A IIh8(;Q[1_V]^Hx/0*>>A~5:*44dLMM r))(L"G=> >Fqy7<0$&99 LL  2== :"K6,-HQ"T h4DIDI  47 3 r"cUR(aUS-n[UR5HAn[R"XR - UR - 5S-n[SU5nMC [UR5H"nUS- UR -UR -nM$ UR(a[R"US- 5n[U5$)a( Return the nearest valid length to use with the model so that there is no time steps left over in a convolution, e.g. for all layers, size of the input - kernel_size % stride = 0. Note that input are automatically padded if necessary to ensure that the output has the same length as the input. rr) rr)r_rceilrr4maxrr)rlength_idxs r valid_lengthDemucs.valid_lengthxs == aKFtzz"AYY)9)9 9T[[HIAMFF^F#$CqjDKK/$2B2BBF% ==YYvz*F6{r"cUnURSnUR(a;URSSS9nURSSS9nURSSS9nX%- SU-- nOSnSnUR U5U- n[ R "X'S-XwS-- 45nUR(a[R"USS5n/nURHn U "U5nURU5 M UR(aURU5nURH)n URS5n [X5n U "X+-5nM+ UR(a[R"USS5nX&-U-n[X#5nUR!UR#S5[%UR&5UR(UR#S55nU$)Nr%rT)rkeepdimgh㈵>rr)r&rmeanrGrFpadrjulius resample_fracrr*rrpoprrsizerrr) rmixr-rmonorrGrsavedrrrs r r:Demucs.forwards  >>88480D99T92D((r4(0CdSj)ADC!!&)F2 EE!qj%1*"45 6 ==$$Q1-AllFq A LLO# 99 ! AllF99R=Dt'Dqx A# ==$$Q1-A GdN  " FF166!9c$,,/1D1DaffRj Qr"c>[UR5HJnSHAnSH8nUSUSU3nUSUSU3nXq;dMXa;dM%URU5X'M: MC ML [TU]XS9 g)N)rr)rJrF.z.3.z.2.)strict)r)r_rrload_state_dict) rstaterrabnewoldrs r rDemucs.load_state_dicts$C++ACqS,CCqS,C|(8%*YYs^ ,,% 5r")rr]rrr_rrrrrrrrrr4)r@g@TrrrTTrrrrrrrrTTg?iD()T) r<r=r>r?rrrr:rrArBrCs@r rrsx!" !EY4Y4v.%N 6 6r"r)rtypingtprr+rtorch.nnrrstatesrutilsrr transformerr Moduler rNrYr[rxrr"r rss  $ &#/BII/d ) DBIIDN;%;%|d6RYYd6r"