CopyRight 2012-2014 DS文库版权所有
Generating Sequences With Recurrent Neural Networks
(0 次评价)2953 人阅读0 次下载

 GeneratingSequencesWithRecurrentNeuralNetworks AlexGravesDepartmentofComputerScienceUniversityofToronto graves@cs.toronto.edu Abstract ThispapershowshowLongShort-termMemoryrecurrentneuralnet-workscanbeusedtogeneratecomplexsequenceswithlong-rangestruc-ture,simplybypredictingonedatapointatatime.Theapproachisdemonstratedfortext(wherethedataarediscrete)andonlinehandwrit-ing(wherethedataarereal-valued).Itisthenextendedtohandwritingsynthesisbyallowingthenetworktoconditionitspredictionsonatextsequence.Theresultingsystemisabletogeneratehighlyrealisticcursivehandwritinginawidevarietyofstyles. 1Introduction Recurrentneuralnetworks(RNNs)arearichclassofdynamicmodelsthathavebeenusedtogeneratesequencesindomainsasdiverseasmusic[6,4],text[30]andmotioncapturedata[29].RNNscanbetrainedforsequencegenerationbyprocessingrealdatasequencesonestepatatimeandpredictingwhatcomesnext.Assumingthepredictionsareprobabilistic,novelsequencescanbegener-atedfromatrainednetworkbyiterativelysamplingfromthenetwork’soutputdistribution,thenfeedinginthesampleasinputatthenextstep.Inotherwordsbymakingthenetworktreatitsinventionsasiftheywerereal,muchlikeapersondreaming.Althoughthenetworkitselfisdeterministic,thestochas-ticityinjectedbypickingsamplesinducesadistributionoversequences.Thisdistributionisconditional,sincetheinternalstateofthenetwork,andhenceitspredictivedistribution,dependsonthepreviousinputs.RNNsare‘fuzzy’inthesensethattheydonotuseexacttemplatesfromthetrainingdatatomakepredictions,butrather—likeotherneuralnetworks—usetheirinternalrepresentationtoperformahigh-dimensionalinterpolationbetweentrainingexamples.Thisdistinguishesthemfromn-grammodelsandcompressionalgorithmssuchasPredictionbyPartialMatching[5],whosepre-dictivedistributionsaredeterminedbycountingexactmatchesbetweentherecenthistoryandthetrainingset.Theresult—whichisimmediatelyappar- 1 arXiv:1308.0850v5 [cs.NE] 5 Jun 2014

打分:

0 星

用户评论:

小飞飞
于 2020-07-15 上传

版权及免责声明|RISC-V单片机中文网 |网站地图

GMT+8, 2022-10-2 13:53 , Processed in 0.546705 second(s), 30 queries .

返回顶部