µ¼¶Á£ºÔÚCVPR 2020ÉÏ£¬£¬£¬£¬£¬£¬ÉÌÌÀAutoMLÍŶÓÌá³öµÄÊ׸ö»ùÓÚ²ÎÊýºóÑéÂþÑÜÌØÕ÷À´Ìá¸ßÍøÂç»ú¹¹ËÑË÷ÐÔÄܵÄÊÂÇ飬£¬£¬£¬£¬£¬¸ÃÊÂÇéΪ½â¾öÍøÂç»ú¹¹ËÑË÷ÖпìËÙÊðÀíµÄÓÐÓÃÐÔÎÊÌâÌṩÁËеÄ˼Ð÷¡£¡£¡£¡£¡£¡£¡£Í¬Ê±¸ÃÍŶÓÒ²¹ûÕæÁËÔÚImageNetÊý¾Ý¼¯ÉϵÄÁ½ÖÖ²î±ð³ß´çµÄÄ£×Ó£¬£¬£¬£¬£¬£¬×÷Ϊģ×ӽṹËÑË÷ºÍÉè¼ÆµÄbaseline¡£¡£¡£¡£¡£¡£¡£Ïà±È֮ǰµÄÊÂÇ飬£¬£¬£¬£¬£¬PC-NASÔڽṹËÑË÷ÊðÀíµÄÓÐÓÃÐÔÉÏÓÐÏÔ×ŵÄÌáÉý¡£¡£¡£¡£¡£¡£¡£

ÄîÍ·

¸ÃÎÄÕÂÌṩÁËÒ»¸öÆÊÎöOne-shot²ÎÊýºóÑéÂþÑܵķ½·¨£¬£¬£¬£¬£¬£¬ÔڵóöËÑË÷¿Õ¼äÖб£´æµÄ½á¹¹Ô½¶à²ÎÊýºóÑéÂþÑÜÆ«ÀëµÃÔ½¶àµÄ½áÂۺ󣬣¬£¬£¬£¬£¬Ìá³öÁËÒ»¸ö»ùÓÚ¿Õ¼äÐÞ¼ôµÄNASËã·¨¡£¡£¡£¡£¡£¡£¡£¸ÃÊÂÇéµÄÑо¿ÄîÍ·ÔÚÓÚ£º

1. Ä¿½ñÍøÂç½á¹¹ËÑË÷Ëã·¨£¨NAS£©ÒѾ­¿ÉÒÔÓ¦ÓÃÓ¦Óᣡ£¡£¡£¡£¡£¡£Æä´ó×ÚµÄÅÌËãÁ¿ÐèÇó³ÉΪÁËÄ¿½ñ×î´óµÄÎÊÌâ¡£¡£¡£¡£¡£¡£¡£È»¶ø½â¾öÅÌËãÁ¿ÎÊÌâµÄÒªÁìÏÖÔÚ»¹²»ÍêÉÆ£¬£¬£¬£¬£¬£¬Ê¹Óù²Ïí²ÎÊýµÄ³¬Íø×öÊðÀí³ÉΪÁËÏÖÔÚ½ÏÁ¿ÆÕ±éµÄÑ¡Ôñ¡£¡£¡£¡£¡£¡£¡£µ«ÕâÀàÒªÁìÊÜÏÞÓÚÊðÀíºÍÕæÕæÏàÐεÄÎó²î¡£¡£¡£¡£¡£¡£¡£

2. ƾ֤ÊÓ²ìºÍÆÊÎö£¬£¬£¬£¬£¬£¬¹²Ïí²ÎÊýÔÚѵÁ·Àú³ÌÖлñµÃµÄ²ÎÊýºóÑéÓëµ¥¸öÄ£×Ó×ÔÁ¦ÑµÁ·µÄ²ÎÊýÂþÑÜÓÐÒ»¶¨µÄ¼ÍÂÉ¡£¡£¡£¡£¡£¡£¡£½ÓÏÂÀ´½«ÏêϸÏÈÈݸÃÊÂÇéµÄÊÓ²ìºÍÆÊÎö¡£¡£¡£¡£¡£¡£¡£

ƾ֤±´Ò¶Ë¹Ä£×ÓÑ¡Ôñ£¨ÒÔ¼°Ò»¶¨µÄ¼ÙÉ裩£¬£¬£¬£¬£¬£¬¸ÃÊÂÇé¶Ô¹²ÏíȨÖضԽṹÅÅÐò´øÀ´µÄÓ°Ïì¸ø³öÁËÁ½¸öÊӲ죺

• ¹²ÏíȨÖصIJÎÊýÂþÑÜÓë²»¹²ÏíµÄretrainʱ¿ÌµÄ½á¹¹²ÎÊýÂþÑܵĹØϵ¿ÉÒÔÓÃKLÉ¢¶ÈÀ´È¨ºâ¡£¡£¡£¡£¡£¡£¡£

• ¿É¹©Ñ¡ÔñµÄÄ£×ÓÊýÄ¿ÉÏÉýʱ£¬£¬£¬£¬£¬£¬¸ÃÉ¢¶ÈËæ׎á½á¹¹ÊýÄ¿ÔöÌí¶øÔöÌí¡£¡£¡£¡£¡£¡£¡£

Ïà¹Øϸ½ÚÇë²Î¿¼ÂÛÎÄ

Ëã·¨

ΪÁËÔÚѵÁ·È¨Öع²ÏíµÄͬʱ£¬£¬£¬£¬£¬£¬¼õСȨÖع²ÏíµÄÍøÂç½á¹¹ÊýÄ¿£¬£¬£¬£¬£¬£¬ÎÒÃÇÌá³öÒ»ÖÖѵÁ·Ê±µÄ¿Õ¼äÐÞ¼ôÒªÁì¡£¡£¡£¡£¡£¡£¡£¼ÙÉèÎÒÃÇÏÖÔÚÓÐÒ»¸öÈçÉÏͼ£¨a£©ËùʾµÄ³¬Íø£¬£¬£¬£¬£¬£¬Ã¿Ò»²ãÓµÓÐÒ»¸ömixop£¨mixopÊÇn¸öºòÑ¡opµÄºÏ³Æ£©ÓÚÊǸó¬ÍøÖÐ×ܹ²°üÀ¨n^²ãÊý¸ö²î±ðµÄ½á¹¹¡£¡£¡£¡£¡£¡£¡£ÎªÁ˾ÙÐÐËÑË÷¿Õ¼äµÄÐÞ¼ô£¬£¬£¬£¬£¬£¬ÎÒÃÇÌá³öÁË ²¿·Ö½á¹¹³ØµÄ¿´·¨(partial model pool)¡£¡£¡£¡£¡£¡£¡£¼ÙÉèÍøÂç½á¹¹ÓÐL²ã£¬£¬£¬£¬£¬£¬²¿·Ö½á¹¹³ØÔÚÕû¸öËÑË÷Àú³ÌÖн«»áÓÐL¸ö״̬£¨Stage£©¡£¡£¡£¡£¡£¡£¡£Ã¿´ÓL-1״̬µ½L״̬£¬£¬£¬£¬£¬£¬²¿·Ö½á¹¹³ØÖеÄ×ӽṹµÄ²ãÊý½«ÔöÌí1¡£¡£¡£¡£¡£¡£¡£

Èçͼ£¨b£©£¬£¬£¬£¬£¬£¬´ËʱStage=2£¬£¬£¬£¬£¬£¬ÔÚ²¿·Ö½á¹¹³ØÖÐÓÐÁ½¸ö²ãÊýΪ2µÄ½á¹¹£¬£¬£¬£¬£¬£¬Í¨¹ý½Úµã±àºÅÃüÃûΪ(1,1)ºÍ(0,1)¡£¡£¡£¡£¡£¡£¡£ÎªÁ˾ÙÐÐÖÁStage=3£¬£¬£¬£¬£¬£¬ÎÒÃÇ»áÆÀ¹À2x3¸ö²¿·Ö½á¹¹µÄDZÁ¦£¬£¬£¬£¬£¬£¬»®·ÖÊÇ£¨1£¬£¬£¬£¬£¬£¬1£¬£¬£¬£¬£¬£¬ 0£©£¬£¬£¬£¬£¬£¬£¨1£¬£¬£¬£¬£¬£¬1£¬£¬£¬£¬£¬£¬1£©£¬£¬£¬£¬£¬£¬£¨1£¬£¬£¬£¬£¬£¬1£¬£¬£¬£¬£¬£¬2£©£¬£¬£¬£¬£¬£¬ÒÔ¼°£¨0£¬£¬£¬£¬£¬£¬1£¬£¬£¬£¬£¬£¬0£©£¬£¬£¬£¬£¬£¬£¨0£¬£¬£¬£¬£¬£¬1£¬£¬£¬£¬£¬£¬1£©£¬£¬£¬£¬£¬£¬£¨0£¬£¬£¬£¬£¬£¬1£¬£¬£¬£¬£¬£¬2£©¡£¡£¡£¡£¡£¡£¡£ÔõÑùÆÀ¼Û²¿·ÖÍøÂçµÄÓÅÁÓ»òÕß˵DZÁ¦ÄØ£¿£¿£¿£¿£¿

Ϊ´ËÎÒÃǽç˵ Potential ÊÇÔÚ²¿·Ö½á¹¹ÖÐΪָ¶¨µÄmixop¾ÙÐÐËæ»ú²ÉÑùµÄ½á¹¹µÄƽ¾ùvalidation set acc¡£¡£¡£¡£¡£¡£¡£

ËÑË÷Ч¹û

¸ü¾ßÉÏÉÏÊöËÑË÷Àú³ÌÎÒÃÇ»®·ÖÔÚÒ»´óһСÁ½¸öÍøÂç¿Õ¼äÖоÙÐÐËÑË÷¡£¡£¡£¡£¡£¡£¡£Ð¡ËÑË÷¿Õ¼äÖÐÿ²ã°üÀ¨9ÖÖoperator£¬£¬£¬£¬£¬£¬´óËÑË÷¿Õ¼ä°üÀ¨Ã¿²ã19ÖÖoperator¡£¡£¡£¡£¡£¡£¡£ÎÒÃÇ°Ñ´óËÑË÷¿Õ¼äµÄËÑË÷Ч¹ûÃüÃûΪPC-NAS-L£¬£¬£¬£¬£¬£¬Ð¡ËÑË÷¿Õ¼äµÄÄ£×ÓÃüÃûΪPC-NAS-S¡£¡£¡£¡£¡£¡£¡£

Éϱí¿ÉÒÔ¿´³ö£¬£¬£¬£¬£¬£¬ÔÚÒ»ÂɵÄgpu lantencyÏ£¬£¬£¬£¬£¬£¬¹þ¹þ(haha)ÌåÓýPC-NAS¿ÉÒÔÁè¼Ý»ò²»µÍÓÚËùÓÐbaseline¡£¡£¡£¡£¡£¡£¡£

ÔÚ¼ì²âʹÃüÖпÉÒÔ¿´Ñé֤ģ×ӵij°ôÐÔ¡£¡£¡£¡£¡£¡£¡£

ÊðÀí±£ÐòÐÔ

ÄÇôÎÒÃǽÓÄɵÄËÑË÷¿Õ¼äËõ¶ÌËã·¨ÊÇ·ñÓÐÓõÄÌá¸ßÁËÊðÀíµÄÓÐÓÃÐÔÄØ£¿£¿£¿£¿£¿

ÎÒÃǽ«Í³Ò»Ì×´úÂ룬£¬£¬£¬£¬£¬Ê¹ÓÃÖØÐÂÖÁβ½ÓÄÉone-shotÒªÁìѵÁ·ºÍʹÓùþ¹þ(haha)ÌåÓýË㷨ѵÁ·µÄshareweights¾ÙÐÐÁ˽ÏÁ¿¡£¡£¡£¡£¡£¡£¡£¹ØÓÚ×îºó±£´æÔÚ×îºóÒ»¸östageÖеIJ¿·Ö½á¹¹Ä£×ÓËùÍØÕ¹µÄ½á¹¹£¬£¬£¬£¬£¬£¬ÉÐÓÐÎÒÃÇËæ»úÔÚone-shotÒªÁìÖвɵÄÏàͬÊýÄ¿µÄ½á¹¹£¬£¬£¬£¬£¬£¬ÕâÁ½¸öȺÌåÖ®¼ä£¬£¬£¬£¬£¬£¬ÕæʵaccºÍproxyaccÖ®¼äµÄÏà¹ØÐÔ¡£¡£¡£¡£¡£¡£¡£ÎÒÃÇ¿ÉÒÔ¿´µ½£¬£¬£¬£¬£¬£¬ÔÚ×îºó½×¶Î£¬£¬£¬£¬£¬£¬Ïà¹ØÐÔ±Èone-shotÒªÁìÒªÓÐÒ»¶¨ÓÅÊÆ¡£¡£¡£¡£¡£¡£¡£

´«ËÍÃÅ

ÂÛÎĵص㣺http://xxx.itp.ac.cn/pdf/1910.02543v1

References

[1] Georege Adam and Jonathan Lorraine. Understanding neural architecture search techniques. arXiv preprint arXiv:1904.00438, 2019.

[2] Bowen Baker, Otkrist Gupta, Nikhil Naik, and Ramesh Raskar. Designing neural network architectures using reinforcement learning. International Conference on Learning Representations, 2017a.

[3] Gabriel Bender, Pieter-Jan Kindermans, Barret Zoph, Vijay Vasudevan, and Quoc V. Le. Understanding and simplifying one-shot architecture search. ICML, 2018.