Sign Up

Have an account? Sign In Now

Sign In

Forgot Password?

Don't have account, Sign Up Here

Forgot Password

Lost your password? Please enter your email address. You will receive a link and will create a new password via email.

Have an account? Sign In Now

Sorry, you do not have a permission to ask a question, You must login to ask question.

Forgot Password?

Need An Account, Sign Up Here
Sign InSign Up

ErrorCorner

ErrorCorner Logo ErrorCorner Logo

ErrorCorner Navigation

  • Home
  • Contact Us
  • About Us
Search
Ask A Question

Mobile menu

Close
Ask a Question
  • Home
  • Contact Us
  • About Us
Home/ Questions/Q 585
Next
Answered
Kenil Vasani
Kenil Vasani

Kenil Vasani

  • 646 Questions
  • 567 Answers
  • 77 Best Answers
  • 26 Points
View Profile
  • 2
Kenil Vasani
Asked: December 14, 20202020-12-14T21:06:00+00:00 2020-12-14T21:06:00+00:00In: Python

Jupyter Notebook (only) Memory Error, same code run in a conventional .py and works

  • 2

I have an assignment for a Deep Learning class, and they provide a Jupyter notebook as a base code, the thing is that after running the data import and reshape, jupyter notebook through a “Memory Error”, after some analysis y tried to compile the same code in a normal .py file, and everything runs well.

The thing is that I’m required (preferably) to use the Jupyter notebook as the base for development, since is more interactive for the kind of task.

<ipython-input-2-846f80a40ce2> in <module>()
  2 # Load the raw CIFAR-10 data
  3 cifar10_dir = 'datasets\\'
----> 4 X, y = load_CIFAR10(cifar10_dir)

C:\path\data_utils.pyc in load_CIFAR10(ROOT)
     18     f = os.path.join(ROOT, 'cifar10_train.p')
     19     print('Path:  ' + f );
---> 20     Xtr, Ytr = load_CIFAR_batch(f)
     21     return Xtr, Ytr
     22 

C:\path\data_utils.pyc in load_CIFAR_batch(filename)
     10         X = np.array(datadict['data'])
     11         Y = np.array(datadict['labels'])
---> 12         X = X.reshape(-1, 3, 32, 32).transpose(0,2,3,1).astype("float")
     13         return X, Y
     14 

MemoryError: 

The error occurs in the line 12, i know is a memory consuming assignment, but that doesn’t mean that 4 GB of RAM wont suffice, and that was confirmed when the code run without problems outside Jupyter.

My Guess is it has something to do with the memory limit either by Jupyter or by Chrome, but I’m not sure and also dont know how to solve it.

By the way:

  • I have a Windows 10 laptop with 4GB of RAM
  • and Chrome Version 57.0.2987.133 (64-bit)
deep-learningjupyterjupyter-consolejupyter-notebookpython
  • 1 1 Answer
  • 11 Views
  • 0 Followers
  • 0
Answer
Share
  • Facebook

    1 Answer

    • Voted
    1. Kenil Vasani

      Kenil Vasani

      • 646 Questions
      • 567 Answers
      • 77 Best Answers
      • 26 Points
      View Profile
      Best Answer
      Kenil Vasani
      2020-12-14T21:02:23+00:00Added an answer on December 14, 2020 at 9:02 pm

      I am only a year and 2 months late to this question. The technical answer as to why is really nicely explained here: https://superuser.com/questions/372881/is-there-a-technical-reason-why-32-bit-windows-is-limited-to-4gb-of-ram

      It also implies why the conda solution works.

      But for a lazy engineer’s no-change workaround, close the Chrome tabs not absolutely necessary and restart your kernel so it starts afresh.

      Kernel > Restart (& Run All)
      
      • 7
      • Share
        Share
        • Share on Facebook
        • Share on Twitter
        • Share on LinkedIn
        • Share on WhatsApp

    You must login to add an answer.

    Forgot Password?

    Sidebar

    Ask A Question
    • Popular
    • Kenil Vasani

      SyntaxError: invalid syntax to repo init in the AOSP code

      • 5 Answers
    • Kenil Vasani

      runtimeError: package fails to pass a sanity check for numpy ...

      • 3 Answers
    • Kenil Vasani

      xlrd.biffh.XLRDError: Excel xlsx file; not supported

      • 3 Answers
    • Kenil Vasani

      Homebrew fails on MacOS Big Sur

      • 3 Answers
    • Kenil Vasani

      SQLSTATE[HY000]: General error: 1835 Malformed communication packet on LARAVEL

      • 2 Answers

    Explore

    • Most Answered
    • Most Visited
    • Most Voted
    • Random

    © 2020-2021 ErrorCorner. All Rights Reserved
    by ErrorCorner.com