]i&SrSSKrSSKJr SSKrSSKrSSKJr SSKr SSK r SSK J r J r SSKJr SSKJrJr \ R&rS rS r"S S \5r"S S\5rS/SSSS.S\R2S\ R4\\4S\ R8\S\ R:\ S\ R:\ R4\\4S\4 Sjjrg)a* DecoratedMain is the main abstraction used inside Dora. This defines the core interface that needs to be fullfilled for Dora to be able to handle an experiment. This is also where the end user will specify how to launch job, default Slurm configuration, storage location, naming conventions etc. N) OrderedDict)Path) DoraConfig SlurmConfig) NamesMixin)XP_contextclURSS5up[R"U5n[X25$)N.r)rsplit importlib import_modulegetattr) full_name module_namefun_namemodules F/mnt/rpi/tmp/demucs-venv-sys/lib/python3.13/site-packages/dora/main.py _load_mainr s2%,,S!4K  $ $[ 1F 6 $$clUS:Xa-[RURnUcgUR$U$)N__main__)sysmodules__spec__name)rspecs rget_module_namer&s5 {{6"++ <99  rc\rSrSrSrSrS\S\4SjrSr S\ R\ S \ 4S jrS rS rS\ R"\ S \4S jrS \ R\ 4SjrS\4SjrS\ S \ R"\ 4SjrS\ S \4SjrSrS\ R2S \ R\ 4SjrS\S \ R\4SjrS \4SjrSrg) DecoratedMain1z Main function that will actually run the XP, wrapped with tons of meta-informations. Args: main (callable, taking no argument): main function for the experiment as provided by the user. dora (DoraConfig): configuration for Dora. Fmaindorac4XlX l[UR5nUcSUlSUlO3SU;a UR SS5uUlUlO SUlX0lURUlURS-UR-Ul g)Nunknowntrainr r) r#r$r __module__package main_moduler r__name__ _full_name)selfr#r$rs r__init__DecoratedMain.__init__<s  %doo6  %DL&D k!1<1C1CC1K. d.( #. LL //C/$--?rc4UR5nURU5(dUR5$URU5nUR U5 [ R "U5 UR5sSSS5 $!,(df  g=fN) _get_argv _is_active_mainget_xpinit_xpr enter_xp)r-argvxps r__call__DecoratedMain.__call__Osf~~t$$::<  [[  R   r "::<# " "s /B  Br8returncgNTr-r8s rr3DecoratedMain._is_activeXsrc"UR5$r1r#r-s rr4DecoratedMain._main[syy{rc([UR44$r1)rr,rDs r __reduce__DecoratedMain.__reduce__^sDOO---rc[5e)z0Return an XP given a list of arguments. NotImplementedErrorr@s rr5DecoratedMain.get_xpas "##rc([RSS$)Nr)rr8rDs rr2DecoratedMain._get_argvfsxx|rr9cURRSSS9 [R"UR[ UR S55 URbURRRRSSSS9 URRRSSSS9 URRRRS5 URRRS5 [R"UR[ URS55 URRS5 U$U$![a Naf=f![a U$f=f)a7 Initialize the XP folder. Once this is done, the XP can be retrieved from its signature. This is done automatically before running, or when using the `--init` flag of the `dora grid` command. This will also initialize the shared XP folder so that the XP hyper-params can be easily shared using its signature. T)exist_okparentswi)rPrQmode) foldermkdirjsondumpr8open _argv_cache_shared_argv_cacheparentchmodPermissionErrorr-r9s rr6DecoratedMain.init_xpls; t4 "''445 ,  ! ! ( ( / / 5 5tTX] 5 ^  ! ! ( ( . .dQV . W %%,,3399%@%%,,2259 IIbggtB$9$93? @ %%++E2 r #   #   s%2AE<E+ E('E(+ E98E9sigc[XRS/S9nURR5(a)[R "[ UR55$URbHURR5(a)[R "[ UR55$[SU35e)zwReturns the argv used to obtain a given signature. This can only work if an XP was previously ran with this signature. N)r`r$cfgr8z)Could not find experiment with signature ) r r$rYexistsrVloadrXrZ RuntimeError)r-r`r9s rget_argv_from_sigDecoratedMain.get_argv_from_sigsCiiT ; >> " "99T"..12 2  " " .23H3H3O3O3Q3Q99T""7"789 9!J3%PQ QrcBURURU55$)zSReturns the XP from the signature. Can only work if such an XP has previously ran. )r5rf)r-r`s rget_xp_from_sigDecoratedMain.get_xp_from_sigs{{411#677rc"SURS3$)NzDecoratedMain()rCrDs r__repr__DecoratedMain.__repr__s {!,,rargc[5e)zConvert a Python value to argv. arg can be either: - a list, then each entry will be converted and all argv are concatenated. - a str, then it is directly an argv entry. - a dict, then each key, value pair is mapped to an argv entry. rJ)r-ros r value_to_argvDecoratedMain.value_to_argvs "##rcbURR5 URR$)zReturn the metrics for a given XP. By default this will look into the `history.json` file, that can be populated with the Link class. Can be overriden, but metrics should still be returned as a list of dicts, possibly with nested dicts. )linkrdhistoryr^s rget_xp_historyDecoratedMain.get_xp_historys  wwrc[5$zEReturn default Slurm config for the launch and grid actions. )rrDs rget_slurm_configDecoratedMain.get_slurm_configs }r)r,r$r#r*rr)N) r+r( __qualname____firstlineno____doc___slowMainFunrr.r:tpListstrboolr3r4rGSequencer r5r2r6rfrirmAnyrqdictrvrrz__static_attributes__r?rrr!r!1s E@W@J@& rwws|.$2;;s+$$ 2773< "6 RS RR[[-= R83828 -$$BGGCL$ +rr!c ^\rSrSrSrSS\S\S\RS\ R\ S\ 4 U4Sjjjr S \ R\S \4S jrS \ R$S \ R&\4S jrS\S \4SjrS \ 4U4SjjrSrU=r$) ArgparseMainaImplementation of `DecoratedMain` for XP that uses argparse. Args: main : main function to wrap. dora : Dora config, containing the exclude and dir fields. parser : parser to use, and to derive default values from. slurm : default slurm config for scheduling jobs. use_underscore : if False, scheduling a job as `launcher(batch_size=32)` will translate to the command-line `--batch-size=32`, otherwise, it will stay as `--batch_size=32`. r#r$parserslurmuse_underscorecH>[TU]X5 X0lXPlX@lgr1)superr.rrr)r-r#r$rrr __class__s rr.ArgparseMain.__init__s  $ , rr8r<c,[U5nURRU5n/nURR 5H8upEURR U5U:wdM&UR XE45 M: [URX!US9nU$)N)r$rbr8delta) listr parse_args__dict__items get_defaultappendr r$)r-r8argsrkeyvaluer9s rr5ArgparseMain.get_xps{Dz{{%%d+----/JC{{&&s+u4 c\*0TYYD5 A rroc/n[U[5(aURU5 U$[U[5(apUR 5HZup4UR (dUR SS5nUSLaURSU35 MCURSUSU35 M\ U$[U[[45(aUHnX RU5- nM U$[SU35e)N_-Tz--=z5Can only process dict, tuple, lists and str, but got ) isinstancerrrrrreplacertuplerq ValueError)r-ror8rrparts rrqArgparseMain.value_to_argvs c3   KK  T " "!iik **++c3/CD=KK"SE +KK"SE5' 23 * dE] + +**400 TUXTYZ[ [rr9cj[5nURceURH up4XBU'M U$r1)rr)r-r9partsrrs rget_name_partsArgparseMain.get_name_partss5 xx###88KD$K$ rcR>URb UR$[TU] 5$ry)rrrz)r-rs rrzArgparseMain.get_slurm_configs' :: !:: w'))r)rrrr>)r+r(r|r}r~rrargparseArgumentParserrOptionalrrr.rrr r5rrrqrrrzr __classcell__)rs@rrrs QUWJ@W@W K0IM2;;s+BGGCL& *+**rrz ./outputsT)direxcludersharedrrrrrrrc <^^^^^^^S[4UUUUUUU4SjjnU$)aNicer version of `ArgparseMain` that acts like a decorator, and directly exposes the most useful configs to override. Args: parser: parser to use, and to derive default values from. exclude: list of patterns of arguments to exclude from the computation of the XP signature. dir: path to store logs, checkpoints, etc. to. slurm: default slurm config for scheduling jobs. shared: path to the shared XP repository. use_underscore: if False, scheduling a job as `launcher(batch_size=32)` will translate to the command-line `--batch-size=32`, otherwise, it will stay as `--batch_size=32`. **kwargs: extra args are passed to `DoraConfig`. r#cz>[S[T5TcSO [T5[T5S.TD6n[XTTTS9$)N)rrr)rrr?)rrrr) r#r$rrkwargsrrrrs r _decorator!argparse_main.._decorator sKS !>4tF|M  D~UZ[[r)r)rrrrrrrrs``````` r argparse_mainrs,\\\ r) r~r collectionsrrrVpathlibrtypingrrconfrrnamesrr9r r Callablerrrr!rrUnionrrrrrr?rrrs#  ) ++% @J@F<*=<*@.9.048=A)- (11xxT *;;s+[1++bhhsDy&9:  #' r