You can use sp_configure to set the max server memory:

  sp_configure 'max server memory', 100
  reconfigure

This will set the server to use at most 100 MB for the buffer cache.

However, you should not do this, because it's perfectly normal for SQL
Server to all memory it can if it needs it. This is because it thinks it
good to have data in cache rather than on disk.

Apparently, you SQL Server instance gets to do some heavy work already on
startup, and that is your real problem. You need to find out what that is.

The following script, will list every user, system, and tempdb table
contained in SQL Server's buffer memory. It is based on a much simpler script
I originally found in a blog somewhere years ago, but I made modifications to
resolve the actual object names and span across all databases. It's pretty
rough, so if you discover something is amiss, like not calculating the
storage MB accurately, then reply back with bug fixes.

-- SQL Server stores its data in 8KB data pages.  As these pages are read
off disk they are stored in memory.  
-- This is referred to as buffer memory.  A list of all the data pages in
memory is stored in the dynamic management
-- view sys.dm_os_buffer_descriptors. Remember that keeping a clustered
index in the buffer (or memory) is the same
-- as keeping the table in memory.
-- NOTE: For performance reasons, I am first inserting the result of the
DMVs to table variables and then joining them.  
declare @sys_allocation_units table
(
    database_id            int,
    allocation_unit_id    bigint,
    type                tinyint,
    type_desc            varchar(20),
    container_id        bigint,
    data_space_id        tinyint,
    total_pages            int,
    used_pages            int,
    data_pages            int
)

insert into @sys_allocation_units
EXEC sp_MSforeachDB @command1='use ?; select db_id(''?'') as database_id, *
from sys.allocation_units'

declare @sys_partitions table
(
    database_id            int,
    objname                varchar(255),
    partition_id        bigint,
    object_id            bigint,
    index_id            int,
    partition_number    int,
    hobt_id                bigint,
    rows                bigint
)

insert into @sys_partitions
EXEC sp_MSforeachDB @command1='use ?; select db_id(''?'') as database_id,
object_name(object_id) as objname, * from sys.partitions;'

declare @allocation table
(
    database_id            int,
    objname                varchar(255),
    index_id            tinyint,
    allocation_unit_id    bigint,
    object_id            bigint
)

insert into @allocation
SELECT au.database_id, objname, index_id ,allocation_unit_id, object_id
FROM @sys_allocation_units AS au
   INNER JOIN @sys_partitions AS p
       ON p.database_id = au.database_id and
            au.container_id = p.hobt_id and
            (au.type = 1 OR au.type = 3)
UNION ALL
SELECT au.database_id, objname, index_id, allocation_unit_id, object_id
FROM @sys_allocation_units AS au
   INNER JOIN @sys_partitions AS p
       ON p.database_id = au.database_id and
            au.container_id = p.hobt_id and
           au.type = 2

declare @bd table
(
    database_id            int,
    allocation_unit_id    bigint,
    Buffered_Page_Count    int,
    row_count            int
)

insert into @bd
SELECT
    bd.database_id,
    bd.allocation_unit_id,
    count(*)AS Buffered_Page_Count ,
    sum(bd.row_count) as row_count
FROM
    sys.dm_os_buffer_descriptors AS bd
GROUP BY
    bd.database_id,
    bd.allocation_unit_id

SELECT --distinct
    db_name(bd.database_id) as DBName,
    obj.ObjName,
    sum(bd.Buffered_Page_Count) as Buffered_Page_Count,
    sum(bd.row_count) as row_count,
    convert(numeric(9,2),sum(bd.Buffered_Page_Count) * 8192.0 / (1024.0 *
1024.0)) as Buffer_MB
FROM
    @bd as bd
   INNER JOIN @allocation AS obj
       ON bd.allocation_unit_id = obj.allocation_unit_id and
            bd.database_id = obj.database_id
group by
    db_name(bd.database_id),
    obj.ObjName
order by
    convert(numeric(9,2),sum(bd.Buffered_Page_Count) * 8192.0 / (1024.0 *
1024.0)) desc
    --db_name(bd.database_id),
    --obj.ObjName

Because your server only has 2GB of memory, and by default SQL Server will
take 2GB of memory - it is taking all of the servers memory.  You need to
set the max memory to something less than 1.25GB of memory which will leave
enough memory for the OS and other SQL Server operations.

Do not set the min memory setting to anything at all - that way the OS can
manage all of the memory that is available.

Jeff