skip to main content
10.1145/3133956.3134056acmconferencesArticle/Chapter ViewAbstractPublication PagesccsConference Proceedingsconference-collections
research-article

Oblivious Neural Network Predictions via MiniONN Transformations

Published: 30 October 2017 Publication History

Abstract

Machine learning models hosted in a cloud service are increasingly popular but risk privacy: clients sending prediction requests to the service need to disclose potentially sensitive information. In this paper, we explore the problem of privacy-preserving predictions: after each prediction, the server learns nothing about clients' input and clients learn nothing about the model.
We present MiniONN, the first approach for transforming an existing neural network to an oblivious neural network supporting privacy-preserving predictions with reasonable efficiency. Unlike prior work, MiniONN requires no change to how models are trained. To this end, we design oblivious protocols for commonly used operations in neural network prediction models. We show that MiniONN outperforms existing work in terms of response latency and message sizes. We demonstrate the wide applicability of MiniONN by transforming several typical neural network models trained from standard datasets.

Supplemental Material

MP4 File

References

[1]
Martín Abadi et al. 2016. Deep Learning with Differential Privacy. In Proceedings of the 2016 ACM SIGSAC Conference on Computer and Communications Security (CCS '16). ACM, New York, NY, USA, 308--318. https://doi.org/10.1145/2976749.2978318
[2]
Martín Abadi et al. 2016. TensorFlow: A System for Large-Scale Machine Learning. In 12th USENIX Symposium on Operating Systems Design and Implementation (OSDI 16). USENIX Association, GA, 265--283. https://www.usenix.org/ conference/osdi16/technical-sessions/presentation/abadi
[3]
Eliana Angelini, Giacomo di Tollo, and Andrea Roli. 2008. A neural network approach for credit risk evaluation. The quarterly review of economics and finance 48, 4 (2008), 733--755.
[4]
Louis J. M. Aslett, Pedro M. Esperança, and Chris C. Holmes. 2015. Encrypted statistical machine learning: new privacy preserving methods. CoRR abs/1508.06845 (2015). http://arxiv.org/abs/1508.06845
[5]
Louis J. M. Aslett, Pedro M. Esperança, and Chris C. Holmes. 2015. A review of homomorphic encryption and software tools for encrypted statistical machine learning. CoRR abs/1508.06574 (2015). http://arxiv.org/abs/1508.06574
[6]
Mauro Barni et al. 2009. Secure Evaluation of Private Linear Branching Programs with Medical Applications. In Computer Security - ESORICS 2009, 14th European Symposium on Research in Computer Security. Springer, Saint-Malo, France, 424--439. http://dx.doi.org/10.1007/978-3-642-04444-1_26
[7]
M. Barni, C. Orlandi, and A. Piva. 2006. A Privacy-preserving Protocol for Neuralnetwork-based Computation. In Proceedings of the 8th Workshop on Multimedia and Security (MM&Sec '06). ACM, New York, NY, USA, 146--151. https://doi.org/ 10.1145/1161366.1161393
[8]
Donald Beaver. 1991. Efficient Multiparty Protocols Using Circuit Randomization. In Advances in Cryptology - CRYPTO '91, 11th Annual International Cryptology Conference, Santa Barbara, California, USA, August 11--15, 1991, Proceedings (Lecture Notes in Computer Science), Vol. 576. Springer, 420--432. https: doi.org/10.1007/3-540-46766-1_34
[9]
James Bergstra et al. 2010. Theano: A CPU and GPU math compiler in Python. In Proc. 9th Python in Science Conf. 1--7.
[10]
Christopher M. Bishop. 2006. Pattern Recognition and Machine Learning (Information Science and Statistics). Springer-Verlag New York, Inc., Secaucus, NJ, USA.
[11]
Dan Bogdanov, Roman Jagomägis, and Sven Laur. 2012. A Universal Toolkit for Cryptographically Secure Privacy-preserving Data Mining. In Proceedings of the 2012 Pacific Asia Conference on Intelligence and Security Informatics (PAISI'12). Springer-Verlag, Berlin, Heidelberg, 112--126. https://doi.org/10.1007/ 978-3-642-30428-6_9
[12]
Dan Bogdanov, Sven Laur, and Jan Willemson. 2008. Sharemind: A Framework for Fast Privacy-Preserving Computations. Springer Berlin Heidelberg, Berlin, Heidelberg, 192--206. https://doi.org/10.1007/978-3-540-88313-5_13
[13]
Joppe W. Bos, Kristin Lauter, Jake Loftus, and Michael Naehrig. 2013. Improved Security for a Ring-Based Fully Homomorphic Encryption Scheme. Springer Berlin Heidelberg, Berlin, Heidelberg, 45--64. https://doi.org/10.1007/978-3-642-45239-0_4
[14]
Raphael Bost, Raluca Ada Popa, Stephen Tu, and Shafi Goldwasser. 2015. Machine Learning Classification over Encrypted Data. In 22nd Annual Network and Distributed System Security Symposium, NDSS 2015, San Diego, California, USA, February 8-11, 2015. http://www.internetsociety.org/doc/ machine-learning-classification-over-encrypted-data
[15]
Justin Brickell, Donald E. Porter, Vitaly Shmatikov, and Emmett Witchel. 2007. Privacy-preserving remote diagnostics. In Proceedings of the 2007 ACM Conference on Computer and Communications Security, CCS 2007, Alexandria, Virginia, USA, October 28-31, 2007. 498--507. http://doi.acm.org/10.1145/1315245.1315307
[16]
Hervé Chabanne, Amaury de Wargny, Jonathan Milgram, Constance Morel, and Emmanuel Prouff. 2017. Privacy-Preserving Classification on Deep Neural Network. Cryptology ePrint Archive, Report 2017/035. (2017). http://eprint.iacr.org/2017/035.
[17]
Jia-Ren Chang and Yong-Sheng Chen. 2015. Batch-normalized Maxout Network in Network. CoRR abs/1511.02583 (2015). http://arxiv.org/abs/1511.02583
[18]
Kumar Chellapilla, Sidd Puri, and Patrice Simard. 2006. High performance convolutional neural networks for document processing. In Tenth International Workshop on Frontiers in Handwriting Recognition. Suvisoft.
[19]
Dan Ciregan, Ueli Meier, and Jürgen Schmidhuber. 2012. Multi-column deep neural networks for image classification. In Computer Vision and Pattern Recognition (CVPR), 2012 IEEE Conference on. IEEE, 3642--3649.
[20]
G. E. Dahl, D. Yu, L. Deng, and A. Acero. 2012. Context-Dependent PreTrained Deep Neural Networks for Large-Vocabulary Speech Recognition. IEEE Transactions on Audio, Speech, and Language Processing 20, 1 (Jan 2012), 30--42. https://doi.org/10.1109/TASL.2011.2134090
[21]
Daniel Demmler, Thomas Schneider, and Michael Zohner. 2015. ABY-A Framework for Efficient Mixed-Protocol Secure Two-Party Computation. In 22nd Annual Network and Distributed System Security Symposium, NDSS 2015, San Diego, California, USA, February 8-11, 2015.
[22]
Paul Dierckx. 1995. Curve and surface fitting with splines. Oxford University Press.
[23]
Nathan Dowlin, Ran Gilad-Bachrach, Kim Laine, Kristin Lauter, Michael Naehrig, and John Wernsing. 2015. Manual for using homomorphic encryption for bioinformatics. Microsoft Research (2015).
[24]
Taher ElGamal. 1985. A Public Key Cryptosystem and a Signature Scheme Based on Discrete Logarithms. In CRYPTO (LNCS), Vol. 196. Springer, 10--18. https://doi.org/10.1007/3-540-39568-7_2
[25]
Rasool Fakoor, Faisal Ladhak, Azade Nazi, and Manfred Huber. 2013. Using deep learning to enhance cancer diagnosis and classification. In Proceedings of the International Conference on Machine Learning.
[26]
Matthew Fredrikson, Eric Lantz, Somesh Jha, Simon Lin, David Page, and Thomas Ristenpart. 2014. Privacy in Pharmacogenetics: An End-to-End Case Study of Personalized Warfarin Dosing. In 23rd USENIX Security Symposium (USENIX Security 14). USENIX Association, 17--32. https://www.usenix.org/conference/usenixsecurity14/technical-sessions/presentation/fredrikson_matthew
[27]
Arik Friedman and Assaf Schuster. 2010. Data Mining with Differential Privacy. In Proceedings of the 16th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining (KDD '10). ACM, New York, NY, USA, 493--502. https://doi.org/10.1145/1835804.1835868
[28]
Ran Gilad-Bachrach, Nathan Dowlin, Kim Laine, Kristin Lauter, Michael Naehrig, and John Wernsing. 2016. CryptoNets: Applying neural networks to encrypted data with high throughput and accuracy. In Proceedings of The 33rd International Conference on Machine Learning. 201--210.
[29]
O. Goldreich, S. Micali, and A. Wigderson. 1987. How to Play ANY Mental Game. In Proceedings of the Nineteenth Annual ACM Symposium on Theory of Computing (STOC '87). ACM, New York, NY, USA, 218--229. https://doi.org/10.1145/28395.28420
[30]
Ian Goodfellow, Yoshua Bengio, and Aaron Courville. 2016. Deep Learning. MIT Press. http://www.deeplearningbook.org.
[31]
Thore Graepel, Kristin E. Lauter, and Michael Naehrig. 2012. ML Confidential: Machine Learning on Encrypted Data. In Information Security and Cryptology - ICISC 2012 - 15th International Conference, Seoul, Korea, November 28-30, 2012, Revised Selected Papers. 1--21. http://dx.doi.org/10.1007/978-3-642-37682-5_1
[32]
Benjamin Graham. 2014. Fractional Max-Pooling. CoRR abs/1412.6071 (2014). http://arxiv.org/abs/1412.6071
[33]
Sepp Hochreiter and Jürgen Schmidhuber. 1997. Long short-term memory. Neural computation 9, 8 (1997), 1735--1780.
[34]
Eric Jones, Travis Oliphant, P Peterson, et al. 2001. SciPy: Open source scientific tools for Python. (2001).
[35]
Nicola Jones. 2014. Nature: Computer science: The learning machines. (2014). http://www.nature.com/news/computer-science-the-learning-machines-1. 14481.
[36]
Alex Krizhevsky and Geoffrey Hinton. 2009. Learning multiple layers of features from tiny images. (2009). http://citeseerx.ist.psu.edu/viewdoc/download?doi=10. 1.1.222.9220&rep=rep1&type=pdf.
[37]
Alex Krizhevsky, Ilya Sutskever, and Geoffrey E Hinton. 2012. ImageNet Classification with Deep Convolutional Neural Networks. In Advances in Neural Information Processing Systems 25, F. Pereira, C. J. C. Burges, L. Bottou, and K. Q. Weinberger (Eds.). Curran Associates, Inc., 1097--1105. http://papers.nips.cc/paper/ 4824-imagenet-classification-with-deep-convolutional-neural-networks.pdf
[38]
Yann LeCun, Corinna Cortes, and Christopher JC Burges. 1998. The MNIST database of handwritten digits. (1998). http://yann.lecun.com/exdb/mnist/.
[39]
Chen-Yu Lee, Patrick W. Gallagher, and Zhuowen Tu. 2016. Generalizing Pooling Functions in Convolutional Neural Networks: Mixed, Gated, and Tree. In Proceedings of the 19th International Conference on Artificial Intelligence and Statistics, AISTATS 2016, Cadiz, Spain, May 9-11, 2016. 464--472. http://jmlr.org/proceedings/papers/v51/lee16a.html
[40]
Dong C Liu and Jorge Nocedal. 1989. On the limited memory BFGS method for large scale optimization. Mathematical programming 45, 1 (1989), 503--528.
[41]
Mitchell P Marcus, Mary Ann Marcinkiewicz, and Beatrice Santorini. 1993. Building a large annotated corpus of English: The Penn Treebank. Computational linguistics 19, 2 (1993), 313--330.
[42]
Tomáš Mikolov et al. 2012. Subword language modeling with neural networks. (2012). http://www.fit.vutbr.cz/imikolov/rnnlm/char.pdf
[43]
Dmytro Mishkin and Jiri Matas. 2015. All you need is a good init. CoRR abs/1511.06422 (2015). http://arxiv.org/abs/1511.06422
[44]
Payman Mohassel and Yupeng Zhang. 2017. SecureML: A System for Scalable Privacy-Preserving Machine Learning. In IEEE Symposium on Security and Privacy (S&P'17). IEEE. http://ieeexplore.ieee.org/document/7958569.
[45]
Kevin P Murphy. 2012. Machine learning: a probabilistic perspective. MIT press.
[46]
Olga Ohrimenko, Felix Schuster, Cedric Fournet, Aastha Mehta, Sebastian Nowozin, Kapil Vaswani, and Manuel Costa. 2016. Oblivious Multi-Party Machine Learning on Trusted Processors. In 25th USENIX Security Symposium (USENIX Security 16). USENIX Association, Austin, TX, 619--636. https://www.usenix.org/conference/usenixsecurity16/technical-sessions/presentation/ohrimenko
[47]
C. Orlandi, A. Piva, and M. Barni. 2007. Oblivious Neural Network Computing via Homomorphic Encryption. EURASIP J. Inf. Secur. 2007, Article 18 (Jan. 2007), 10 pages. https://doi.org/10.1155/2007/37343
[48]
Pascal Paillier. 1999. Public-Key Cryptosystems Based on Composite Degree Residuosity Classes. In EUROCRYPT (LNCS), Jacques Stern (Ed.), Vol. 1592. Springer, 223--238. https://doi.org/10.1007/3-540-48910-X_16
[49]
Pille Pullonen and Sander Siim. 2015. Combining Secret Sharing and Garbled Circuits for Efficient Private IEEE 754 Floating-Point Computations. In Financial Cryptography and Data Security - FC 2015 International Workshops, BITCOIN, WAHC, and Wearable, San Juan, Puerto Rico, January 30, 2015, Revised Selected Papers. 172--183. https://doi.org/10.1007/978-3-662-48051-9_13
[50]
Bita Darvish Rouhani, M. Sadegh Riazi, and Farinaz Koushanfar. 2017. DeepSecure: Scalable Provably-Secure Deep Learning. CoRR abs/1705.08963 (2017). http://arxiv.org/abs/1705.08963
[51]
Ikuro Sato, Hiroki Nishimura, and Kensuke Yokoi. 2015. APAC: Augmented PAttern Classification with Neural Networks. CoRR abs/1505.03229 (2015). http://arxiv.org/abs/1505.03229
[52]
Reza Shokri and Vitaly Shmatikov. 2015. Privacy-Preserving Deep Learning. In Proceedings of the 22Nd ACM SIGSAC Conference on Computer and Communications Security (CCS '15). ACM, New York, NY, USA, 1310--1321. https://doi.org/10.1145/2810103.2813687
[53]
Reza Shokri, Marco Stronati, Congzheng Song, and Vitaly Shmatikov. 2017. Membership Inference Attacks Against Machine Learning Models. In 2017 IEEE Symposium on Security and Privacy, SP 2017, San Jose, CA, USA, May 22-26, 2017. IEEE, 3--18. https://doi.org/10.1109/SP.2017.41
[54]
N. P. Smart and F. Vercauteren. 2014. Fully homomorphic SIMD operations. Designs, Codes and Cryptography 71, 1 (2014), 57--81. https://doi.org/10.1007/s10623-012-9720-4
[55]
Jost Tobias Springenberg, Alexey Dosovitskiy, Thomas Brox, and Martin A. Riedmiller. 2014. Striving for Simplicity: The All Convolutional Net. CoRR abs/1412.6806 (2014). http://arxiv.org/abs/1412.6806
[56]
Florian Tramèr, Fan Zhang, Ari Juels, Michael K. Reiter, and Thomas Ristenpart. 2016. Stealing Machine Learning Models via Prediction APIs. In 25th USENIX Security Symposium (USENIX Security 16). USENIX Association, Austin, TX, 601--618. https://www.usenix.org/conference/usenixsecurity16/technical-sessions/presentation/tramer
[57]
Li Wan, Matthew Zeiler, Sixin Zhang, Yann L. Cun, and Rob Fergus. 2013. Regularization of Neural Networks using DropConnect. In Proceedings of the 30th International Conference on Machine Learning (ICML-13), Sanjoy Dasgupta and David Mcallester (Eds.). JMLR Workshop and Conference Proceedings, 1058--1066. http://jmlr.org/proceedings/papers/v28/wan13.pdf
[58]
David J. Wu, Tony Feng, Michael Naehrig, and Kristin E. Lauter. 2016. Privately Evaluating Decision Trees and Random Forests. Privacy Enhancing Technologies (PoPETs) 2016, 4 (2016), 335--355. http://dx.doi.org/10.1515/popets-2016-0043
[59]
Andrew Chi-Chih Yao. 1982. Protocols for Secure Computations (Extended Abstract). In Foundations of Computer Science (FOCS'82). IEEE, 160--164.
[60]
Andrew C.-C. Yao. 1986. How to Generate and Exchange Secrets. In Foundations of Computer Science (FOCS'86). IEEE, 162--167.
[61]
Wojciech Zaremba, Ilya Sutskever, and Oriol Vinyals. 2014. Recurrent neural network regularization. CoRR abs/1409.2329 (2014). http://arxiv.org/abs/1409.2329

Cited By

View all
  • (2025)Circuit Privacy for FHEW/TFHE-Style Fully Homomorphic Encryption in PracticeIACR Communications in Cryptology10.62056/av11c3w9p1:4Online publication date: 13-Jan-2025
  • (2025)Machine Learning Meets Encrypted Search: The Impact and Efficiency of OMKSA in Data SecurityInternational Journal of Intelligent Systems10.1155/int/24295772025:1Online publication date: 16-Jan-2025
  • (2025)SEPPDL: A Secure and Efficient Privacy-Preserving Deep Learning Inference Framework for Autonomous DrivingACM Transactions on Autonomous and Adaptive Systems10.1145/3708505Online publication date: 9-Jan-2025
  • Show More Cited By

Index Terms

  1. Oblivious Neural Network Predictions via MiniONN Transformations

    Recommendations

    Comments

    Information & Contributors

    Information

    Published In

    cover image ACM Conferences
    CCS '17: Proceedings of the 2017 ACM SIGSAC Conference on Computer and Communications Security
    October 2017
    2682 pages
    ISBN:9781450349468
    DOI:10.1145/3133956
    Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than the author(s) must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected].

    Sponsors

    Publisher

    Association for Computing Machinery

    New York, NY, United States

    Publication History

    Published: 30 October 2017

    Permissions

    Request permissions for this article.

    Check for updates

    Author Tags

    1. machine learning
    2. neural network predictions
    3. privacy
    4. secure two-party computation

    Qualifiers

    • Research-article

    Funding Sources

    Conference

    CCS '17
    Sponsor:

    Acceptance Rates

    Overall Acceptance Rate 382 of 2,124 submissions, 18%

    Upcoming Conference

    CCS '25

    Contributors

    Other Metrics

    Bibliometrics & Citations

    Bibliometrics

    Article Metrics

    • Downloads (Last 12 months)186
    • Downloads (Last 6 weeks)24
    Reflects downloads up to 30 Jan 2025

    Other Metrics

    Citations

    Cited By

    View all
    • (2025)Circuit Privacy for FHEW/TFHE-Style Fully Homomorphic Encryption in PracticeIACR Communications in Cryptology10.62056/av11c3w9p1:4Online publication date: 13-Jan-2025
    • (2025)Machine Learning Meets Encrypted Search: The Impact and Efficiency of OMKSA in Data SecurityInternational Journal of Intelligent Systems10.1155/int/24295772025:1Online publication date: 16-Jan-2025
    • (2025)SEPPDL: A Secure and Efficient Privacy-Preserving Deep Learning Inference Framework for Autonomous DrivingACM Transactions on Autonomous and Adaptive Systems10.1145/3708505Online publication date: 9-Jan-2025
    • (2025)Panther: Practical Secure Two-Party Neural Network InferenceIEEE Transactions on Information Forensics and Security10.1109/TIFS.2025.352606320(1149-1162)Online publication date: 2025
    • (2025)Single-Layer Trainable Neural Network for Secure InferenceIEEE Internet of Things Journal10.1109/JIOT.2024.348019512:3(2968-2978)Online publication date: 1-Feb-2025
    • (2025)GuardianMPC: Backdoor-Resilient Neural Network ComputationIEEE Access10.1109/ACCESS.2025.352830413(11029-11048)Online publication date: 2025
    • (2025)Privacy-preserving and verifiable convolution neural network inference and training in cloud computingFuture Generation Computer Systems10.1016/j.future.2024.107560164(107560)Online publication date: Mar-2025
    • (2025)Towards practical and privacy-preserving CNN inference service for cloud-based medical imaging analysis: A homomorphic encryption-based approachComputer Methods and Programs in Biomedicine10.1016/j.cmpb.2025.108599261(108599)Online publication date: Apr-2025
    • (2025)A Faster Privacy-Preserving Medical Image Diagnosis Scheme with Machine LearningJournal of Imaging Informatics in Medicine10.1007/s10278-024-01384-4Online publication date: 3-Jan-2025
    • (2024)Accelerating secure collaborative machine learning with protocol-aware RDMAProceedings of the 33rd USENIX Conference on Security Symposium10.5555/3698900.3699026(2245-2261)Online publication date: 14-Aug-2024
    • Show More Cited By

    View Options

    Login options

    View options

    PDF

    View or Download as a PDF file.

    PDF

    eReader

    View online with eReader.

    eReader

    Figures

    Tables

    Media

    Share

    Share

    Share this Publication link

    Share on social media