[mpich-discuss] Running an mpi program that needs to access /dev/mem
Antonio J. Peña
apenya at mcs.anl.gov
Fri Jun 14 11:57:30 CDT 2013
I've created ticket #1885 to check this issue. Here it's the URL:
http://trac.mpich.org/projects/mpich/ticket/1885
Antonio
On Friday, June 14, 2013 04:18:06 PM Lee, Eibhlin wrote:
> Thank you Antonio.
>
> http://www.skpang.co.uk/blog/archives/615 If you follow the steps to
> download adc you will get 4 files that need to be included in compilation:
> gb_common.h gb_spi.h gb_common.c and gb_spi.c I have included the files
> here as well.
>
> I compile them using
> mpicc example_for_MPICH.c gb_common.c gb_spi.c examp -lm -Lbbcm2835-1.25/src
> -lbcm2835 and execute in the standard manner for smpd but this will be
> different in hydra. I normally run on 2 machines and have 2 processes
> started.
>
> If you could please reply with the output in full that would help me. (I do
> NEED /dev/mem it is directly accessed in gb_common.h)
>
> Thanks,
> Eibhlin
>
> ________________________________________
> From: discuss-bounces at mpich.org [discuss-bounces at mpich.org] on behalf of
> Antonio J. Peña [apenya at mcs.anl.gov] Sent: 14 June 2013 16:10
> To: discuss at mpich.org
> Subject: Re: [mpich-discuss] Running an mpi program that needs to
> access /dev/mem
>
> Hi Eibhlin,
>
> If you share a piece of code to test your issue, I can run it for you using
> hydra to check if that solves it.
>
> Antonio
>
> On Friday, June 14, 2013 11:27:25 AM Lee, Eibhlin wrote:
> > I found that the reason we want to access /dev/mem is to setup memory
> > regions to access the peripherals. (We are trying to read the output of an
> > ADC). At this point it becomes more a linux/raspberry-pi specific problem
> > than an MPICH problem. Although the fact that you can't run a program that
> > needs access to memory mapping (even as the root user) seems something
> > that
> > MPICH could improve on for future versions. I know I am using smpd instead
> > of hydra so this problem may already be solved. But if someone could
> > confirm that, it would be really helpful.
> > ________________________________________
> > From: discuss-bounces at mpich.org [discuss-bounces at mpich.org] on behalf of
> > Lee, Eibhlin [eibhlin.lee10 at imperial.ac.uk] Sent: 14 June 2013 11:20
> > To: discuss at mpich.org
> > Subject: Re: [mpich-discuss] Running an mpi program that needs to
> >
> > access /dev/mem
> >
> > Gus,
> > I tried running cpi, as is included in the installation of MPI, on two
> > machines with two processes. The output message confirmed that it had
> > started only 1 process instead of 2. Process 0 of 1 is on raspi
> > pi is approximately...
> >
> > Then it just hung. I think this is because the other machine didn't know
> > where to output the data?
> >
> > When I tried running two processes on the one machine using the wrapper
> > you
> > suggested the output was the same but doubled. It didn't hang. This
> > confirms that every process was started with rank 0.
> >
> > I'm not entirely sure why /dev/mem is needed. I'm working in a group and
> > another member set up io and gpio and it seemed it needed access to
> > /dev/mem I am going to do a strace as suggested by Pavan Balaji to see
> > where it is used and see if I can somehow work around it.
> >
> > Thank you for your help.
> > Eibhlin
> > ________________________________________
> > From: discuss-bounces at mpich.org [discuss-bounces at mpich.org] on behalf of
> > Gus Correa [gus at ldeo.columbia.edu] Sent: 13 June 2013 21:11
> > To: Discuss Mpich
> > Subject: Re: [mpich-discuss] Running an mpi program that needs to
> > access /dev/mem
> >
> > Hi Eibhlin
> >
> > On 06/13/2013 12:59 PM, Lee, Eibhlin wrote:
> > > Gus,
> > > I believe your first assumption is correct. Unfortunately it just seemed
> > > to hang. I think this might be because each one is being made to have
> > > the
> > > same rank...
> >
> > Darn! I was afraid that it might give only rank 0 to all MPI processes.
> > So, with the script wrapper the process being launched by mpiexec may
> > indeed be sudo,
> > not the actual mpi executable (main) :(
> > Then it may actually launch a bunch of separate rank 0 replicas of your
> > program,
> > instead of assigning to them different ranks.
> > However, without any output or error message, it is hard to tell.
> >
> > No output at all?
> > No error message, just hangs?
> > Have you tried a verbose flag (-v) to mpiexec?
> > (Not sure if it exists in MPICH mpiexec, you'd need to check.)
> >
> > Would you care to try it with another mpi program,
> > one that doesn't deal with /dev/mem (a risky business),
> > say cpi.c (in the examples directory), or an mpi version of Hello, world,
> > just to see if the mpiexec+sudo_script_wrapper works as expected or
> > if everybody gets rank 0?
> >
> > > It may already be obvious but this is the first time I am using Linux. I
> > > had tried sudo $(which mpiexec ....) and sudo $(which mpiexec) ... both
> > > without success.
> >
> > "which mpiexec" will return the path to mpiexec, but won't execute it.
> >
> > You could try this (with backquotes):
> >
> > `which mpiexec` -n 2 ~/main
> >
> > On a side note, make sure the mpiexec you're using matches the
> > mpicc/mpif90/MPI library from the MPICH that
> > you used to compile the program.
> > Often times computers have several flavors of MPI installed, and mixing
> > them just doesn't work.
> >
> > > Is putting the full path to it similar to/is a symlink? (This still
> > > doesn't make main have super user privileges though.)
> >
> > No, nothing to do with sudo privileges.
> >
> > This suggestion was just to avoid messing up your /usr/bin,
> > which is a directory that despite the somewhat misleading name (/usr,
> > for historical reasons I think),
> > is supposed to hold system (Linux) programs (that users can use), but
> > not user-installed programs.
> > Normally things are that are installed in /usr get there via some Linux
> > package manager program
> > (yum, rpm, apt-get, etc), to keep consistency with libraries, etc.
> >
> > I belive MPICH would install by default in /usr/local/ (and put mpiexec
> > in /usr/local/bin),
> > which is kind of a default location for non-system applications.
> >
> > The full path suggestion would be something like:
> > /path/to/where/you/installed/mpiexec -n 2 ~/main
> >
> > However, this won't solve the other problem w.r.t. sudo and /dev/mem.
> >
> > You must know what you are doing, but it made me wonder,
> > even if your program were sequential, why would you want to mess with
> > /dev/mem directly?
> > Just curious about it.
> >
> > Gus Correa
> >
> > > Eibhlin
> > > ________________________________________
> > > From: discuss-bounces at mpich.org [discuss-bounces at mpich.org] on behalf of
> > > Gus Correa [gus at ldeo.columbia.edu] Sent: 13 June 2013 15:37
> > > To: Discuss Mpich
> > > Subject: Re: [mpich-discuss] Running an mpi program that needs to
> > > access /dev/mem
> > >
> > > Hi Lee
> > >
> > > How about replacing "~/main" in the mpiexec command line
> > > by one-liner script?
> > > Say, "sudo_main.sh", something like this:
> > >
> > > #! /bin/bash
> > > sudo ~/main
> > >
> > > After all, it is "main" that accesses /dev/mem,
> > > and needs "sudo" permissions, not mpiexec, right?
> > > [Or do the mpiexec-launched processes inherit
> > > the "sudo" stuff from mpiexec?]
> > >
> > > Not related, but, instead of putting mpiexec in /usr/bin,
> > > can't you just use the full path to it?
> > >
> > > I hope this helps,
> > > Gus Correa
> > >
> > > On 06/13/2013 10:09 AM, Lee, Eibhlin wrote:
> > >> Pavan,
> > >> I had a lot of trouble getting hydra to work without having to enter a
> > >> password/passphrase. I saw the option to pass a phrase in the mpich
> > >> installers guide. I eventually found that for that command you needed
> > >> to
> > >> use the smpd process manager. That's the only reason I chose smpd over
> > >> hydra. As to your other suggestion. I ran ./main and the same error
> > >> (Can't open /dev/mem...) appeared. sudo ./main works but of course
> > >> without multiple processes. Eibhlin
> > >> ________________________________________
> > >> From: discuss-bounces at mpich.org [discuss-bounces at mpich.org] on behalf
> > >> of
> > >> Pavan Balaji [balaji at mcs.anl.gov] Sent: 13 June 2013 14:34
> > >> To: discuss at mpich.org
> > >> Subject: Re: [mpich-discuss] Running an mpi program that needs to
> > >> access
> > >>
> > >> /dev/mem
> > >>
> > >> I just saw your older email. Why are you using smpd instead of the
> > >> default process manager (hydra)?
> > >>
> > >> -- Pavan
> > >>
> > >> On 06/13/2013 08:05 AM, Pavan Balaji wrote:
> > >>> What's "-phrase"? That's not a recognized option. I'm not sure where
> > >>> the /dev/mem check is coming from. Try running ~/main without mpiexec
> > >>> first.
> > >>>
> > >>> -- Pavan
> > >>>
> > >>> On 06/13/2013 06:56 AM, Lee, Eibhlin wrote:
> > >>>> Hello all,
> > >>>>
> > >>>> I am trying to use two raspberry-pi to sample and then process some
> > >>>> data. The first process samples while the second processes and vice
> > >>>> versa. To do this I use gpio and also mpich-3.0.4 with the process
> > >>>> manager smpd. I have successfully run cpi on both machines (from the
> > >>>> master machine). I have also managed to run a similar program but
> > >>>> without the MPI, this involved compiling with gcc and when running
> > >>>> putting sudo in front of the binary file.
> > >>>>
> > >>>> When I combine these two processes I get various error messages.
> > >>>> For input:
> > >>>> mpiexec -phrase cat -machinefile machinefile -n 2 ~/main
> > >>>> the error is:
> > >>>> Can't open /dev/mem
> > >>>> Did you forget to use 'sudo .. ?'
> > >>>>
> > >>>> For input:
> > >>>> sudo mpiexec -phrase cat -machinefile machinefile -n 2 ~/main
> > >>>> the error is:
> > >>>> sudo: mpiexec: Command not found
> > >>>>
> > >>>> I therefore put mpiexec into /usr/bin
> > >>>>
> > >>>> now for input:
> > >>>> sudo mpiexec -phrase cat -machinefile machinefile -n 2 ~/main
> > >>>> the error is:
> > >>>> Can't open /dev/mem
> > >>>> Did you forget to use 'sudo .. ?'
> > >>>>
> > >>>> Does anyone know how I can work around this?
> > >>>> Thanks,
> > >>>> Eibhlin
> > >>>>
> > >>>>
> > >>>>
> > >>>> _______________________________________________
> > >>>> discuss mailing list discuss at mpich.org
> > >>>> To manage subscription options or unsubscribe:
> > >>>> https://lists.mpich.org/mailman/listinfo/discuss
> > >>
> > >> --
> > >> Pavan Balaji
> > >> http://www.mcs.anl.gov/~balaji
> > >> _______________________________________________
> > >> discuss mailing list discuss at mpich.org
> > >> To manage subscription options or unsubscribe:
> > >> https://lists.mpich.org/mailman/listinfo/discuss
> > >> _______________________________________________
> > >> discuss mailing list discuss at mpich.org
> > >> To manage subscription options or unsubscribe:
> > >> https://lists.mpich.org/mailman/listinfo/discuss
> > >
> > > _______________________________________________
> > > discuss mailing list discuss at mpich.org
> > > To manage subscription options or unsubscribe:
> > > https://lists.mpich.org/mailman/listinfo/discuss
> > > _______________________________________________
> > > discuss mailing list discuss at mpich.org
> > > To manage subscription options or unsubscribe:
> > > https://lists.mpich.org/mailman/listinfo/discuss
> >
> > _______________________________________________
> > discuss mailing list discuss at mpich.org
> > To manage subscription options or unsubscribe:
> > https://lists.mpich.org/mailman/listinfo/discuss
> > _______________________________________________
> > discuss mailing list discuss at mpich.org
> > To manage subscription options or unsubscribe:
> > https://lists.mpich.org/mailman/listinfo/discuss
> > _______________________________________________
> > discuss mailing list discuss at mpich.org
> > To manage subscription options or unsubscribe:
> > https://lists.mpich.org/mailman/listinfo/discuss
>
> _______________________________________________
> discuss mailing list discuss at mpich.org
> To manage subscription options or unsubscribe:
> https://lists.mpich.org/mailman/listinfo/discuss
More information about the discuss
mailing list